Variational autoencoders employ an amortized inference model to approximate the posterior of latent variables. However, such amortized variational inference faces two challenges: (1) the limited posterior expressiveness of fully-factorized Gaussian assumption and (2) the amortization error of the inference model. We present a novel approach that addresses both challenges. First, we focus on ReLU networks with Gaussian output and illustrate their connection to probabilistic PCA. Building on this observation, we derive an iterative algorithm that finds the mode of the posterior and apply full-covariance Gaussian posterior approximation centered on the mode. Subsequently, we present a general framework named Variational Laplace Autoencoders (VLAEs) for training deep generative models. Based on the Laplace approximation of the latent variable posterior, VLAEs enhance the expressiveness of the posterior while reducing the amortization error. Empirical results on MNIST, Omniglot, Fashion-MNIST, SVHN and CIFAR10 show that the proposed approach significantly outperforms other recent amortized or iterative methods on the ReLU networks.
翻译:自动自动解析器采用一个摊销式推论模型,以近似潜在变量的后部。然而,这种摊销式变差推论面临着两个挑战:(1) 完全因素化高斯假设的后部外表表现有限,(2) 推算模型的摊销错误。我们提出了一个应对这两个挑战的新办法。首先,我们侧重于带有高斯输出输出的ReLU网络,并展示其与概率性五氯苯的关联。在此观察的基础上,我们产生了一种迭代算法,找到后部模式,并在此模式中采用完全变量化高斯后部近距离近距离近似。随后,我们提出了一个名为Variational Laplace Auterencoders(VLAE)的一般框架,用于培训深层基因模型。基于潜伏变量后部后部近地点的近似近似近似近似,VLAEE加强了后部的外观感光度方法,同时减少了沉积错误。在MNIST、Omniglot、FAS-FAR10或远端系统上展示最新高亮的图像系统模型、SLIS-HAFADISADRADRADR的系统上展示其他新式方法。