Recent work in unsupervised learning has focused on efficient inference and learning in latent variables models. Training these models by maximizing the evidence (marginal likelihood) is typically intractable. Thus, a common approximation is to maximize the Evidence Lower BOund (ELBO) instead. Variational autoencoders (VAE) are a powerful and widely-used class of generative models that optimize the ELBO efficiently for large datasets. However, the VAE's default Gaussian choice for the prior imposes a strong constraint on its ability to represent the true posterior, thereby degrading overall performance. A Gaussian mixture model (GMM) would be a richer prior, but cannot be handled efficiently within the VAE framework because of the intractability of the Kullback-Leibler divergence for GMMs. We challenge the adoption of the VAE framework on this specific point in favor of one with an analytical solution for Gaussian mixture prior. To perform efficient inference for GMM priors, we introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. This new objective allows us to incorporate richer, multi-modal priors into the auto-encoding framework.We provide empirical studies on a range of datasets and show that our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
翻译:未经监督的近期学习工作侧重于潜在变量模型中高效的推断和学习。 通过最大限度地增加证据(边缘可能性)来培训这些模型通常难以解决。 因此,一个共同近似点是最大限度地增加证据下库(ELBO), 从而在 VAE 中, 动态自动读数器(VAE) 是一个强大和广泛使用的基因模型类别, 能高效优化ELBO, 用于大型数据集。 然而, VAE 的默认 Gaussian 选择前一种数据, 对其代表真实的后代人的能力施加了强烈的制约, 从而降低了总体性业绩。 高斯混合模型(GMMM) 之前会更丰富一些, 但却无法在 VAE 框架内高效处理。 由于 Kullback- Leiber 差异对于 GMMM 来说是不可忽视的。 我们在此特定点上采用 VAE 框架, 有利于高斯 之前的混合物的分析解决方案。 为 GM 之前的快速推导力, 我们引入了基于 Caci- Schwar 模型的新限制目标。