Minimization of the (regularized) entropy of classification probabilities is a versatile class of discriminative clustering methods. The classification probabilities are usually defined through the use of some classical losses from supervised classification and the point is to avoid modelisation of the full data distribution by just optimizing the law of the labels conditioned on the observations. We give the first theoretical study of such methods, by specializing to logistic classification probabilities. We prove that if the observations are generated from a two-component isotropic Gaussian mixture, then minimizing the entropy risk over a Euclidean ball indeed allows to identify the separation vector of the mixture. Furthermore, if this separation vector is sparse, then penalizing the empirical risk by a $\ell_{1}$-regularization term allows to infer the separation in a high-dimensional space and to recover its support, at standard rates of sparsity problems. Our approach is based on the local convexity of the logistic entropy risk, that occurs if the separation vector is large enough, with a condition on its norm that is independent from the space dimension. This local convexity property also guarantees fast rates in a classical, low-dimensional setting.
翻译:最小化分类概率(正规化)的概率最小化是一个多用途的差别组合方法类别。分类概率通常通过使用监督分类中某些古典损失来界定。分类概率通常通过使用监督分类中的某些典型损失来界定,其要点是避免仅仅通过优化以观察为条件的标签法来模拟全部数据分布的模型化。我们对这种方法进行首次理论研究,专门研究后勤分类概率。我们证明,如果观测来自两种成分的异质高斯混合物,然后将欧洲二氯丁二烯球的酶风险最小化,从而确实能够确定混合物的分离矢量。此外,如果这种分离矢量稀少,然后用一个$\ell ⁇ 1}(美元)-常规化术语来惩罚经验风险,从而可以推断在高空间的分离,并以标准速度恢复其支持。我们的方法是以物流酶风险的本地共性为基础,如果分离矢量足够大,如果分离矢量的矢量足够大,且其规范的条件独立于空间层面,则会出现这种风险。此外,这种本地的惯性特性也能够快速设定。