Variational Bayes (VB) has become a widely-used tool for Bayesian inference in statistics and machine learning. Nonetheless, the development of the existing VB algorithms is so far generally restricted to the case where the variational parameter space is Euclidean, which hinders the potential broad application of VB methods. This paper extends the scope of VB to the case where the variational parameter space is a Riemannian manifold. We develop an efficient manifold-based VB algorithm that exploits both the geometric structure of the constraint parameter space and the information geometry of the manifold of VB approximating probability distributions. Our algorithm is provably convergent and achieves a convergence rate of order $\mathcal O(1/\sqrt{T})$ and $\mathcal O(1/T^{2-2\epsilon})$ for a non-convex evidence lower bound function and a strongly retraction-convex evidence lower bound function, respectively. We develop in particular two manifold VB algorithms, Manifold Gaussian VB and Manifold Neural Net VB, and demonstrate through numerical experiments that the proposed algorithms are stable, less sensitive to initialization and compares favourably to existing VB methods.
翻译:在统计学和机器学习方面,变异贝雅斯(VB)已成为一种广泛使用的巴伊西亚推算工具,然而,现有的VB算法的开发迄今一般限于变异参数空间为Euclidean的情况,这妨碍了VB方法的潜在广泛应用。本文将VB的范围扩大到变异参数空间为Riemannian 方块的情况。我们开发了一种高效的基于多个的VB算法,既利用制约参数空间的几何结构,又利用VB相近概率分布的方块的信息几何方法。我们的算法可以令人接受地趋同,并实现了美元/gathcal O(1/\sqrt{T}) 和 $\mathcal O(1/T ⁇ 2\\\\epsilon} 的趋同率。我们开发了一种基于非convex证据的较低约束功能的高效的VB算法。我们开发了两种特别多VB算法,Manfound Galian VB 和Manfold Nevelyal 的初始实验方法, 通过VnalB 和Manxalalalalalal 模拟的模拟算法显示了现有的Vxal 和较不具有较不稳定的数值。