大多数概率模型中, 计算后验边际或准确计算归一化常数都是很困难的. 变分推断(variational inference)是一个近似计算这两者的框架. 变分推断把推断看作优化问题: 我们尝试根据某种距离度量来寻找一个与真实后验尽可能接近的分布(或者类似分布的表示).

VIP内容

概率图模型的形式化提供了一个统一的框架,以捕获随机变量之间的复杂依赖关系,并建立大规模的多元统计模型。图模型已经成为许多统计、计算和数学领域的研究重点,包括生物信息学、通信理论、统计物理、组合优化、信号和图像处理、信息检索和统计机器学习。在特定情况下出现的许多问题——包括计算边际和概率分布模式的关键问题——最好在一般情况下进行研究。利用指数族表示,并利用指数族的累积函数和熵之间的共轭对偶性,我们发展了计算似然、边际概率和最可能配置问题的一般变分表示。我们描述了各种各样的算法——其中包括和积、聚类变分方法、期望传播、平均场方法、最大积和线性规划松弛,以及圆锥规划松弛——是如何以这些变分表示的精确或近似形式来理解的。变分方法提供了一个补充替代马尔可夫链蒙特卡罗作为一个一般来源的逼近方法推断在大规模统计模型。

https://www.nowpublishers.com/article/Details/MAL-001

成为VIP会员查看完整内容
0
32

最新论文

Non-linear hierarchical models are commonly used in many disciplines. However, inference in the presence of non-nested effects and on large datasets is challenging and computationally burdensome. This paper provides two contributions to scalable and accurate inference. First, I derive a new mean-field variational algorithm for estimating binomial logistic hierarchical models with an arbitrary number of non-nested random effects. Second, I propose "marginally augmented variational Bayes" (MAVB) that further improves the initial approximation through a step of Bayesian post-processing. I prove that MAVB provides a guaranteed improvement in the approximation quality at low computational cost and induces dependencies that were assumed away by the initial factorization assumptions. I apply these techniques to a study of voter behavior using a high-dimensional application of the popular approach of multilevel regression and post-stratification (MRP). Existing estimation took hours whereas the algorithms proposed run in minutes. The posterior means are well-recovered even under strong factorization assumptions. Applying MAVB further improves the approximation by partially correcting the under-estimated variance. The proposed methodology is implemented in an open source software package.

0
0
下载
预览
Top