This paper introduces the $f$-divergence variational inference ($f$-VI) that generalizes variational inference to all $f$-divergences. Initiated from minimizing a crafty surrogate $f$-divergence that shares the statistical consistency with the $f$-divergence, the $f$-VI framework not only unifies a number of existing VI methods, e.g. Kullback-Leibler VI, R\'{e}nyi's $\alpha$-VI, and $\chi$-VI, but offers a standardized toolkit for VI subject to arbitrary divergences from $f$-divergence family. A general $f$-variational bound is derived and provides a sandwich estimate of marginal likelihood (or evidence). The development of the $f$-VI unfolds with a stochastic optimization scheme that utilizes the reparameterization trick, importance weighting and Monte Carlo approximation; a mean-field approximation scheme that generalizes the well-known coordinate ascent variational inference (CAVI) is also proposed for $f$-VI. Empirical examples, including variational autoencoders and Bayesian neural networks, are provided to demonstrate the effectiveness and the wide applicability of $f$-VI.
翻译:本文介绍了“美元-美元-美元-六”的“美元-美元-六”的波动变率推导法(f-美元-六),该法将变差推导至所有美元-美元-美元-振幅。从尽量减少与美元-振幅(f-digence)具有统计一致性的巧妙代用价(f-美元-振幅)开始,美元-VI框架不仅统一了现有的若干六种方法,例如“Kullback-Leibel VI”、“R\'{{{e}nii's dalpha$-六”和“$-chi$-六”等,但为六国提供了一个标准化的“六国”适用性工具包,但与美元-美元-振幅家族有任意的分歧。“美元-美元-振幅”的通用代用工具是“美元-美元-波动约束”的,并提供了边际可能性(或证据)的三明治估计。美元-六国-六国-六国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国-国