We consider a class of nonsmooth optimization problems over the Stiefel manifold, in which the objective function is weakly convex in the ambient Euclidean space. Such problems are ubiquitous in engineering applications but still largely unexplored. We present a family of Riemannian subgradient-type methods -- namely Riemannain subgradient, incremental subgradient, and stochastic subgradient methods -- to solve these problems and show that they all have an iteration complexity of ${\cal O}(\varepsilon^{-4})$ for driving a natural stationarity measure below $\varepsilon$. In addition, we establish the local linear convergence of the Riemannian subgradient and incremental subgradient methods when the problem at hand further satisfies a sharpness property and the algorithms are properly initialized and use geometrically diminishing stepsizes. To the best of our knowledge, these are the first convergence guarantees for using Riemannian subgradient-type methods to optimize a class of nonconvex nonsmooth functions over the Stiefel manifold. The fundamental ingredient in the proof of the aforementioned convergence results is a new Riemannian subgradient inequality for restrictions of weakly convex functions on the Stiefel manifold, which could be of independent interest. We also show that our convergence results can be extended to handle a class of compact embedded submanifolds of the Euclidean space. Finally, we discuss the sharpness properties of various formulations of the robust subspace recovery and orthogonal dictionary learning problems and demonstrate the convergence performance of the algorithms on both problems via numerical simulations.
翻译:我们考虑的是Stiefel 元体上的非摩擦优化问题, 其目标功能在周围的 Euclidea 空间中是微弱的, 使自然稳定度测量低于$\calepsilon$。 这些问题在工程应用中是无处不在的, 但仍然基本上没有探索。 我们提出一个里曼尼亚梯级方法的组合, 即 Riemannain 亚梯度、 递增的亚梯度和随机亚梯度方法, 以解决这些问题, 并表明它们都具有 $=cal O} (\varepsilon) (\ varepsilon) 的循环复杂性复杂性。 此外, 我们建立了里曼亚的亚梯度和递增的亚梯度方法的局部趋同, 当问题进一步满足了尖锐的属性, 并使用了几度递减的阶梯度方法。 据我们所知, 这些是使用里曼亚次梯度亚的亚梯度亚梯度的分位化方法, 优化非星级的不相趋同级的稳定性函数, 也显示我们Steriglegalalalalalalallial lial listallistal listal listal listal listal listal ligal