We consider the problem of minimizing a non-convex function over a smooth manifold $\mathcal{M}$. We propose a novel algorithm, the Orthogonal Directions Constrained Gradient Method (ODCGM) which only requires computing a projection onto a vector space. ODCGM is infeasible but the iterates are constantly pulled towards the manifold, ensuring the convergence of ODCGM towards $\mathcal{M}$. ODCGM is much simpler to implement than the classical methods which require the computation of a retraction. Moreover, we show that ODCGM exhibits the near-optimal oracle complexities $\mathcal{O}(1/\varepsilon^2)$ and $\mathcal{O}(1/\varepsilon^4)$ in the deterministic and stochastic cases, respectively. Furthermore, we establish that, under an appropriate choice of the projection metric, our method recovers the landing algorithm of Ablin and Peyr\'e (2022), a recently introduced algorithm for optimization over the Stiefel manifold. As a result, we significantly extend the analysis of Ablin and Peyr\'e (2022), establishing near-optimal rates both in deterministic and stochastic frameworks. Finally, we perform numerical experiments which shows the efficiency of ODCGM in a high-dimensional setting.
翻译:我们考虑的是将非碳化函数在平滑的元元中最小化的问题。 我们提出一种新型算法, 即Orthogoal Directors Contratracted Gradient 方法( ODCGM ), 它只需要计算向矢量空间的投影。 ODCGM 是不可行的, 但是循环体会不断地拉向这个方块, 确保 ODCGM 与 $\ mathcal{M} 。 ODCGM 的实施比 需要计算回调的经典方法( 2022) 更简单得多。 此外, 我们显示 ODCGM 展示了近最佳或最精密的方块复杂性 $( 1/ varepsilon) $ 和 $\ mathcal{O} ( 1/\ varepslon% 4) 。 在确定性和随机分析中, 我们确定, 在适当选择的预测度指标下, 我们的方法可以恢复Ablin 和 Peyr\ e e( 20), 最近引入的对Stiefelfel complace- complia 进行优化的调校正的计算法分析, 我们最后在确定了一个高度框架。</s>