Learning a Gaussian Mixture Model (GMM) is hard when the number of parameters is too large given the amount of available data. As a remedy, we propose restricting the GMM to a Gaussian Markov Random Field Mixture Model (GMRF-MM), as well as a new method for estimating the latter's sparse precision (i.e., inverse covariance) matrices. When the sparsity pattern of each matrix is known, we propose an efficient optimization method for the Maximum Likelihood Estimate (MLE) of that matrix. When it is unknown, we utilize the popular Graphical Least Absolute Shrinkage and Selection Operator (GLASSO) to estimate that pattern. However, we show that even for a single Gaussian, when GLASSO is tuned to successfully estimate the sparsity pattern, it does so at the price of a substantial bias of the values of the nonzero entries of the matrix, and we show that this problem only worsens in a mixture setting. To overcome this, we discard the nonzero values estimated by GLASSO, keep only its pattern estimate and use it within the proposed MLE method. This yields an effective two-step procedure that removes the bias. We show that our "debiasing" approach outperforms GLASSO in both the single-GMRF and the GMRF-MM cases. We also show that when learning priors for image patches, our method outperforms GLASSO even if we merely use an educated guess about the sparsity pattern, and that our GMRF-MM outperforms the baseline GMM on real and synthetic high-dimensional datasets.
翻译:当参数数量过大时, 高山混合模型( GMMM) 很难学习 。 由于可用数据的数量太多, 我们建议将 GMM 限制为高山 Markov 随机字段混合模型( GMRF- MM ), 以及一种新的方法来估计后者的稀疏精度( 即反共变) 矩阵。 当知道每个矩阵的宽度模式时, 我们为该矩阵的最大相似度估计( MLE) 提出了一个有效的优化方法 。 当未知时, 我们使用广受欢迎的最小绝对缩小和选择操作员( GLASSO ) 来估计该模式。 但是, 我们建议将GMMMMM 限制限制到一个普通模式模型模型模型模型模型模型( GRM ), 当GASSO 调整成功估计时, 我们只能使用一种真实的 GASSL 方法, 并且我们用一种有效的方法来显示我们的最低GASSL 。