We show that on the manifold of fixed-rank and symmetric positive semi-definite matrices, the Riemannian gradient descent algorithm almost surely escapes some spurious critical points on the boundary of the manifold. Our result is the first to partially overcome the incompleteness of the low-rank matrix manifold without changing the vanilla Riemannian gradient descent algorithm. The spurious critical points are some rank-deficient matrices that capture only part of the eigen components of the ground truth. Unlike classical strict saddle points, they exhibit very singular behavior. We show that using the dynamical low-rank approximation and a rescaled gradient flow, some of the spurious critical points can be converted to classical strict saddle points in the parameterized domain, which leads to the desired result. Numerical experiments are provided to support our theoretical findings.
翻译:我们发现,在固定的和对称的正正半确定基质中,里曼梯度梯度运算法几乎肯定逃脱了多块线边界上一些虚假的临界点。 我们的结果是首先部分克服低位矩阵的不完整,而不改变香草里曼梯度梯度运算法。 虚假的临界点是某些等级偏差的基点,它只捕捉到地面真伪的部分成分。 与古典的严格马鞍点不同,它们表现出非常奇特的行为。 我们显示,使用动态的低级近似和重新标定梯度流,一些虚假临界点可以转换为参数化域的典型的严格马鞍点,从而得出预期的结果。 提供了数字实验来支持我们的理论结论。