In this paper, we focus on escaping from saddle points in smooth nonconvex optimization problems subject to a convex set $\mathcal{C}$. We propose a generic framework that yields convergence to a second-order stationary point of the problem, if the convex set $\mathcal{C}$ is simple for a quadratic objective function. To be more precise, our results hold if one can find a $\rho$-approximate solution of a quadratic program subject to $\mathcal{C}$ in polynomial time, where $\rho<1$ is a positive constant that depends on the structure of the set $\mathcal{C}$. Under this condition, we show that the sequence of iterates generated by the proposed framework reaches an $(\epsilon,\gamma)$-second order stationary point (SOSP) in at most $\mathcal{O}(\max\{\epsilon^{-2},\rho^{-3}\gamma^{-3}\})$ iterations. We further characterize the overall arithmetic operations to reach an SOSP when the convex set $\mathcal{C}$ can be written as a set of quadratic constraints. Finally, we extend our results to the stochastic setting and characterize the number of stochastic gradient and Hessian evaluations to reach an $(\epsilon,\gamma)$-SOSP.
翻译:在本文中, 我们关注的焦点是, 以平滑的非convex优化问题为主, 平滑的非convex 优化问题 。 我们建议了一个通用框架, 如果 comvex 设置 $\ mathcal{C} 美元对于二次目标功能来说很简单, 那么这个框架会与问题第二阶固定点趋同。 更精确地说, 我们的结果可以维持, 如果在多边时间找到 $\ mathcal{C} (\maxcal{ {C} $) 的四极方案近乎$\ mathcal{ $, $\ rho_ 1$是一个正常数, 取决于 设置 $\ mathcal{C} 的结构。 在此条件下, 我们显示, 由拟议框架生成的转折数序列的顺序会达到$ (\ emsilon,\ gammama) (SOS) 最多是 =3\ gemama_ 3_ 美元。 我们进一步将整个算算算算算算算算算算出我们S- salalalaltical ral rass ral ral rass ration ration a cal end end ends ax end rus a end ends sqalticaltical rus) rus a endaltic) rus rus a rus rus rus a endal rus endalticalticaltic) rus a ex rus rus rus rus rus endal rus endal ral rus ral ral ral ral ral ral ral ends) ends.