We introduce a new variational inference (VI) framework, called energetic variational inference (EVI). It minimizes the VI objective function based on a prescribed energy-dissipation law. Using the EVI framework, we can derive many existing Particle-based Variational Inference (ParVI) methods, including the popular Stein Variational Gradient Descent (SVGD) approach. More importantly, many new ParVI schemes can be created under this framework. For illustration, we propose a new particle-based EVI scheme, which performs the particle-based approximation of the density first and then uses the approximated density in the variational procedure, or "Approximation-then-Variation" for short. Thanks to this order of approximation and variation, the new scheme can maintain the variational structure at the particle level, and can significantly decrease the KL-divergence in each iteration. Numerical experiments show the proposed method outperforms some existing ParVI methods in terms of fidelity to the target distribution.
翻译：我们引入了新的变式推断(VI)框架,称为强变式推断(EVI) 。 它根据规定的能量分散法,将六种目标功能最小化。 使用 EVI 框架, 我们可以得出许多现有的基于粒子的变异推断( ParVI) 方法, 包括流行的Stein variational Egradient Emple (SVGD) 方法。 更重要的是, 在这个框架下可以创建许多新的 ParVI 方案。 举例来说, 我们提出一个新的基于颗粒的 EVI 方案, 首先对密度进行粒基近似, 然后在变异程序中使用大约密度, 或“ 近似同步- 即时变异” 用于短时间。 由于这种近似和变异的顺序, 新的方案可以维持粒子级水平的变异结构, 并大大降低每次迭代的KL- diverence。 数值实验显示, 拟议的方法在对目标分布的精确性方面优于现有的PARVI 方法。
相关内容
微信扫码咨询专知VIP会员