Convex optimization over the spectrahedron, i.e., the set of all real $n\times n$ positive semidefinite matrices with unit trace, has important applications in machine learning, signal processing and statistics, mainly as a convex relaxation for optimization with low-rank matrices. It is also one of the most prominent examples in the theory of first-order methods for convex optimization in which non-Euclidean methods can be significantly preferable to their Euclidean counterparts, and in particular the Matrix Exponentiated Gradient (MEG) method which is based on the Bregman distance induced by the (negative) von Neumann entropy. Unfortunately, implementing MEG requires a full SVD computation on each iteration, which is not scalable to high-dimensional problems. In this work we propose efficient implementations of MEG, both with deterministic and stochastic gradients, which are tailored for optimization with low-rank matrices, and only use a single low-rank SVD computation on each iteration. We also provide efficiently-computable certificates for the correct convergence of our methods. Mainly, we prove that under a strict complementarity condition, the suggested methods converge from a "warm-start" initialization with similar rates to their full-SVD-based counterparts. Finally, we bring empirical experiments which both support our theoretical findings and demonstrate the practical appeal of our methods.
翻译:在光谱优化方面,所有真实的美元-时间正正正半确定基质的组合,在机器学习、信号处理和统计方面都有重要的应用,主要是对低级基质的优化的松动放松,也是对光谱优化第一阶方法理论中最突出的例子之一,在这一理论中,非单子优化方法可以大大优于对等方,特别是基于(负)von Neumann entropy引起的布雷格曼距离(MEG)的矩阵指数指数化渐进(MEG)方法。不幸的是,实施MEG需要在每个迭代上进行完全的SVD计算,而这种计算不能伸缩到高维度问题。在这项工作中,我们建议采用非单次低级SVD方法来优化基于低级基质的矩阵,而只使用单次低级的SVD计算方法。我们还可以在每次外加总基点上提供一种严格的实验性趋同率标准。我们提出的最终检验标准是“在最接近的实验性标准之下,我们最终将一个有效的实验性结果转化为。