A key design constraint when implementing Monte Carlo and variational inference algorithms is that it must be possible to cheaply and exactly evaluate the marginal densities of proposal distributions and variational families. This takes many interesting proposals off the table, such as those based on involved simulations or stochastic optimization. This paper broadens the design space, by presenting a framework for applying Monte Carlo and variational inference algorithms when proposal densities cannot be exactly evaluated. Our framework, recursive auxiliary-variable inference (RAVI), instead approximates the necessary densities using meta-inference: an additional layer of Monte Carlo or variational inference, that targets the proposal, rather than the model. RAVI generalizes and unifies several existing methods for inference with expressive approximating families, which we show correspond to specific choices of meta-inference algorithm, and provides new theory for analyzing their bias and variance. We illustrate RAVI's design framework and theorems by using them to analyze and improve upon Salimans et al.'s Markov Chain Variational Inference, and to design a novel sampler for Dirichlet process mixtures, achieving state-of-the-art results on a standard benchmark dataset from astronomy and on a challenging datacleaning task with Medicare hospital data.
翻译:实施 Monte Carlo 和变异推断算法的一个关键设计制约是,必须能够以廉价和准确的方式评估建议分布和变异家庭的边际密度。这需要许多令人感兴趣的建议,例如基于模拟或随机优化的建议。本文扩大了设计空间,提出了适用Monte Carlo的框架,以及在无法精确评估建议密度时采用变异推断算法。我们的框架,重复的辅助可变推断性推断(RAVI),而不是利用元推法来接近必要的密度:增加一个蒙特卡洛或变异推法层,以提案为目标,而不是模型。RAVI概括并统一了与表情相近家庭的现有推断方法,我们显示了与具体选择的元推断算法和变异性算法的相对应,并为分析其偏差和差异提供了新理论。我们用RAVI的设计框架和理论来分析并改进Salimans Markov Charil Intravely Exciational Brigulational-Climateal-Climateal-Climateal-dal-Crefirmal 数据,我们用它们分析并改进了一种具有挑战性的数据,并设计一个具有挑战性的模型的数据。