Annealed Importance Sampling (AIS) is a popular algorithm used to estimates the intractable marginal likelihood of deep generative models. Although AIS is guaranteed to provide unbiased estimate for any set of hyperparameters, the common implementations rely on simple heuristics such as the geometric average bridging distributions between initial and the target distribution which affect the estimation performance when the computation budget is limited. Optimization of fully parametric AIS remains challenging due to the use of Metropolis-Hasting (MH) correction steps in Markov transitions. We present a parameteric AIS process with flexible intermediary distributions and optimize the bridging distributions to use fewer number of steps for sampling. A reparameterization method that allows us to optimize the distribution sequence and the parameters of Markov transitions is used which is applicable to a large class of Markov Kernels with MH correction. We assess the performance of our optimized AIS for marginal likelihood estimation of deep generative models and compare it to other estimators.
翻译:Annaaled Streature Sampling(AIS)是一种常用的算法,用于估计深基因模型的难处理的边际可能性。虽然AIS保证为任何一套超参数提供无偏差的估计,但通常的执行依赖于简单的超光速学,例如初始和目标分布之间的几何平均桥接力分布,在计算预算有限时会影响估计性能。由于在Markov过渡中使用大都会-Hasting(MH)校正步骤,优化完全参数的AIS仍然具有挑战性。我们提出了一个具有灵活中间分布的参数 AIS进程,并优化桥接分布,以便使用较少的采样步骤。采用了一种再计法,使我们能够优化分布序列和Markov过渡参数,该方法适用于大类Markov Kernels和MH校正。我们评估了优化的AIS的性能,用于对深基因模型进行边际可能性估计,并将其与其他估计器进行比较。