Variational autoencoders model high-dimensional data by positing low-dimensional latent variables that are mapped through a flexible distribution parametrized by a neural network. Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations. Existing approaches to posterior collapse often attribute it to the use of neural networks or optimization issues due to variational approximation. In this paper, we consider posterior collapse as a problem of latent variable non-identifiability. We prove that the posterior collapses if and only if the latent variables are non-identifiable in the generative model. This fact implies that posterior collapse is not a phenomenon specific to the use of flexible distributions or approximate inference. Rather, it can occur in classical probabilistic models even with exact inference, which we also demonstrate. Based on these results, we propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility. This model class resolves the problem of latent variable non-identifiability by leveraging bijective Brenier maps and parameterizing them with input convex neural networks, without special variational inference objectives or optimization tricks. Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.
翻译:自动自动解析器模拟高维数据。 通过假设低维潜伏变量, 这些变量是通过神经网络的灵活分布匹配而绘制的。 不幸的是, 变异自动解析器通常会受到后表崩溃的影响: 潜伏变量的后表与前表相等, 使得变异自动解析器无法作为产生有意义的表达方式。 现有的后表崩溃方法往往将其归因于使用神经网络或因变化近似而产生的优化问题。 在本文中, 我们认为后表崩溃是潜伏变异性不可识别性的问题。 我们证明, 仅当潜在变异在基因化模型中无法识别时, 变异性自动解剖析器会崩溃。 这一事实意味着, 后表崩溃并非是使用灵活分布或近似推断的一种特有的现象。 相反, 后表崩溃可能发生在传统的概率模型中, 甚至是精确的推论。 基于这些结果, 我们提出了一种可辨度变异性变异性变异性变异性变异性自动解变异性, 深度的变异性模型, 在不使变易变的变性变的变现性变变性变的变变性数据中, 提供可变性变性变性变性变性变性变性变性变性变性数据, 和变性变性变性变性变性变性变性数据模型, 以不变性变性变性变的变性变性变性数据 以不变的变的变性变性数据, 以不变的变的变的变性变性数据 变性模型 以不变性变性变的变的变的变的模型 。