We study the variational inference problem of minimizing a regularized R\'enyi divergence over an exponential family, and propose a relaxed moment-matching algorithm, which includes a proximal-like step. Using the information-geometric link between Bregman divergences and the Kullback-Leibler divergence, this algorithm is shown to be equivalent to a Bregman proximal gradient algorithm. This novel perspective allows us to exploit the geometry of our approximate model while using stochastic black-box updates. We use this point of view to prove strong convergence guarantees including monotonic decrease of the objective, convergence to a stationary point or to the minimizer, and geometric convergence rates. These new theoretical insights lead to a versatile, robust, and competitive method, as illustrated by numerical experiments.
翻译:通过Bregman近端梯度算法最小化正则化的Rényi散度。我们研究了在指数族上最小化正则化Rényi散度的变分推断问题,并提出了一种松弛的矩匹配算法,其中包括一步近似法。利用Bregman散度与Kullback-Leibler散度之间的信息几何联系,我们将该算法显示等价于Bregman近端梯度算法。这种新颖的视角使我们能够在使用随机黑盒更新的同时利用近似模型的几何形状。我们利用这种观点证明了强收敛保证,包括目标基元的单调减少,收敛到稳定点或最小化点,以及几何收敛速度。这些新的理论见解导致了一种通用、稳健、竞争力强的方法,如数值实验所示。