We introduce Sparse Variational Bayesian Monte Carlo (SVBMC), a method for fast "post-process" Bayesian inference for models with black-box and potentially noisy likelihoods. SVBMC reuses all existing target density evaluations -- for example, from previous optimizations or partial Markov Chain Monte Carlo runs -- to build a sparse Gaussian process (GP) surrogate model of the log posterior density. Uncertain regions of the surrogate are then refined via active learning as needed. Our work builds on the Variational Bayesian Monte Carlo (VBMC) framework for sample-efficient inference, with several novel contributions. First, we make VBMC scalable to a large number of pre-existing evaluations via sparse GP regression, deriving novel Bayesian quadrature formulae and acquisition functions for active learning with sparse GPs. Second, we introduce noise shaping, a general technique to induce the sparse GP approximation to focus on high posterior density regions. Third, we prove theoretical results in support of the SVBMC refinement procedure. We validate our method on a variety of challenging synthetic scenarios and real-world applications. We find that SVBMC consistently builds good posterior approximations by post-processing of existing model evaluations from different sources, often requiring only a small number of additional density evaluations.
翻译:我们引入了Sparse variational Bayesian Monte Carlo (SVBMC),这是一种快速“后处理” Bayesian 快速推断黑盒和潜在噪音可能性模型的方法。SVBMC重新使用所有现有的目标密度评价 -- -- 例如,从先前的优化或部分的Markov Cain Cain Monte Car 运行中重新使用所有现有的目标密度评价 -- -- 以构建一个稀疏的Gaussian进程(GP) 替代模型来取代日志后传密度。然后根据需要,通过积极学习来改进替代点的不确定区域。我们的工作建立在VBMSian Monte Car(VBMC) 样本高效推断框架上,并作出一些新的贡献。首先,我们通过稀疏的GP回归,将VBMC 推广到大量的预存在的评价,我们通过稀疏的GPGP 生成新的Byesic 公式和获取功能。第二,我们引入了一种一般技术来引导稀疏的GP milling press milling to to at to fore at to fore at to fol potical pract practal resmilling press acal prilling proup produstrevation produstrutal.</s>