Gaussian mixture models are a popular tool for model-based clustering, and mixtures of factor analyzers are Gaussian mixture models having parsimonious factor covariance structure for mixture components. There are several recent extensions of mixture of factor analyzers to deep mixtures, where the Gaussian model for the latent factors is replaced by a mixture of factor analyzers. This construction can be iterated to obtain a model with many layers. These deep models are challenging to fit, and we consider Bayesian inference using sparsity priors to further regularize the estimation. A scalable natural gradient variational inference algorithm is developed for fitting the model, and we suggest computationally efficient approaches to the architecture choice using overfitted mixtures where unnecessary components drop out in the estimation. In a number of simulated and two real examples, we demonstrate the versatility of our approach for high-dimensional problems, and demonstrate that the use of sparsity inducing priors can be helpful for obtaining improved clustering results.
翻译:Gausian 混合物模型是基于模型的组合工具,而系数分析器的混合物是高斯的混合物模型,具有混合物组成部分的相近因因子共变结构。最近有一些因素分析器混合物扩展为深混合物,其中潜伏系数模型被混合要素分析器所取代。这种构造可以迭代,以获得一个多层模型。这些深层模型是难以适应的。我们认为,在进一步规范估计之前,使用孔径推推法,是难以适应的。为了适应模型,我们开发了一种可缩放的自然梯度变异推算法,我们建议使用不必要部件退出估计的过配混合物对结构选择采用计算高效的方法。在一系列模拟和两个真实的例子中,我们展示了我们应对高维问题的方法的多功能性,并表明,使用粘度诱导先期方法可以帮助改进组合结果。