Non-convex sampling is a key challenge in machine learning, central to non-convex optimization in deep learning as well as to approximate probabilistic inference. Despite its significance, theoretically there remain many important challenges: Existing guarantees (1) typically only hold for the averaged iterates rather than the more desirable last iterates, (2) lack convergence metrics that capture the scales of the variables such as Wasserstein distances, and (3) mainly apply to elementary schemes such as stochastic gradient Langevin dynamics. In this paper, we develop a new framework that lifts the above issues by harnessing several tools from the theory of dynamical systems. Our key result is that, for a large class of state-of-the-art sampling schemes, their last-iterate convergence in Wasserstein distances can be reduced to the study of their continuous-time counterparts, which is much better understood. Coupled with standard assumptions of MCMC sampling, our theory immediately yields the last-iterate Wasserstein convergence of many advanced sampling schemes such as proximal, randomized mid-point, and Runge-Kutta integrators. Beyond existing methods, our framework also motivates more efficient schemes that enjoy the same rigorous guarantees.
翻译:非阴道取样是机器学习中的一个关键挑战,在深层学习中非阴道优化的核心,以及近似概率推导中,非阴道取样是关键的挑战。尽管其意义重大,理论上仍然存在许多重大挑战:现有保障(1) 通常只维持平均流星,而不是最理想的最后一次迭代国,(2) 缺乏反映瓦瑟斯坦距离等变量规模的趋同指标,以及(3) 主要适用于诸如随机梯度梯度兰格文动态等初级方案。在本文件中,我们开发了一个新的框架,通过利用动态系统理论中的若干工具来提升上述问题。我们的主要结果是,对于大量最先进的采样计划而言,其瓦塞斯坦距离的最后一种地差可以缩短到研究其连续时间对应数据,这一点更为人们所理解。结合了监控中心采样的标准假设,我们的理论立即产生了许多先进采样计划的最后一种岩层瓦塞尔斯坦趋同,例如准、随机选的中点和Renge-Kutest Intestric 等。除了我们的现有方法之外,还享受了同样有效的保证。</s>