This work identifies the existence and cause of a type of posterior collapse that frequently occurs in the Bayesian deep learning practice. For a general linear latent variable model that includes linear variational autoencoders as a special case, we precisely identify the nature of posterior collapse to be the competition between the likelihood and the regularization of the mean due to the prior. Our result suggests that posterior collapse may be related to neural collapse and dimensional collapse and could be a subclass of a general problem of learning for deeper architectures.
翻译:这项工作确定了在巴伊西亚深层学习实践中经常发生的一种后遗骨崩溃的存在和原因。对于将线性变异自动电解码器作为特例的普通线性潜伏变量模型来说,我们准确地确定后遗骨崩溃的性质是因前一种而导致的平均值的可能性和正规化之间的竞争。我们的结果表明后遗骨崩溃可能与神经崩溃和立体崩溃有关,并可能成为深层建筑一般学习问题的一个亚类。