We consider the geometric ergodicity of the Stochastic Gradient Langevin Dynamics (SGLD) algorithm under nonconvexity settings. Via the technique of reflection coupling, we prove the Wasserstein contraction of SGLD when the target distribution is log-concave only outside some compact set. The time discretization and the minibatch in SGLD introduce several difficulties when applying the reflection coupling, which are addressed by a series of careful estimates of conditional expectations. As a direct corollary, the SGLD with constant step size has an invariant distribution and we are able to obtain its geometric ergodicity in terms of $W_1$ distance. The generalization to non-gradient drifts is also included.
翻译:我们认为Stochatic Gradient Langevin Dynamics(SGLD)算法在非电解设置下具有几何性。 通过反射组合技术,当目标分布仅在某些紧凑装置外的日志集合时,我们证明了SGLD的瓦瑟斯坦缩缩缩。 SGLD的时间分解和小批量在应用反射组合时造成了一些困难,这些困难通过一系列对有条件期望的仔细估计来解决。 直接的推论是,具有恒定步尺寸的SGLD具有一种变异性分布,我们能够以1美元的距离获得其几何偏差。 也包括非梯度漂移的一般化。