Stein variational gradient descent (SVGD) refers to a class of methods for Bayesian inference based on interacting particle systems. In this paper, we consider the originally proposed deterministic dynamics as well as a stochastic variant, each of which represent one of the two main paradigms in Bayesian computational statistics: variational inference and Markov chain Monte Carlo. As it turns out, these are tightly linked through a correspondence between gradient flow structures and large-deviation principles rooted in statistical physics. To expose this relationship, we develop the cotangent space construction for the Stein geometry, prove its basic properties, and determine the large-deviation functional governing the many-particle limit for the empirical measure. Moreover, we identify the Stein-Fisher information (or kernelised Stein discrepancy) as its leading order contribution in the long-time and many-particle regime in the sense of $\Gamma$-convergence, shedding some light on the finite-particle properties of SVGD. Finally, we establish a comparison principle between the Stein-Fisher information and RKHS-norms that might be of independent interest.
翻译:斯坦因梯度变化性下降(SVGD) 指一种基于交互粒子系统的贝耶斯推论方法。 在本文中,我们考虑了最初提议的确定性动态以及一个随机变异,其中每个变异都代表了巴伊西亚计算统计的两个主要范例之一:变异性推论和Markov 链条 Monte Carlo。 事实证明,通过梯度流结构与扎根于统计物理学的大型振荡原则之间的对应,这些都紧密地联系在一起。 为了揭露这种关系,我们为斯坦因几何系统开发了相切空间结构,证明了其基本特性,并确定了关于实证措施多粒度限制的大型降功能。此外,我们确定了斯坦-菲舍信息(或内嵌化的斯坦因子差异)是其长期和多粒子制度中的主要成份贡献,即$\Gamma$-convergence,对SVGD的有限粒子特性有所了解。 最后,我们在斯坦-Fisher信息与RKHS-stoms之间确立了一项可能具有独立利益的比较原则。