Many real-world problems not only have complicated nonconvex functional constraints but also use a large number of data points. This motivates the design of efficient stochastic methods on finite-sum or expectation constrained problems. In this paper, we design and analyze stochastic inexact augmented Lagrangian methods (Stoc-iALM) to solve problems involving a nonconvex composite (i.e. smooth+nonsmooth) objective and nonconvex smooth functional constraints. We adopt the standard iALM framework and design a subroutine by using the momentum-based variance-reduced proximal stochastic gradient method (PStorm) and a postprocessing step. Under certain regularity conditions (assumed also in existing works), to reach an $\varepsilon$-KKT point in expectation, we establish an oracle complexity result of $O(\varepsilon^{-5})$, which is better than the best-known $O(\varepsilon^{-6})$ result. Numerical experiments on the fairness constrained problem and the Neyman-Pearson classification problem with real data demonstrate that our proposed method outperforms an existing method with the previously best-known complexity result.
翻译:许多现实世界问题不仅复杂了非混凝土功能限制,而且还使用大量的数据点。 这促使设计了关于有限和/或预期限制问题的高效随机方法。 在本文中,我们设计和分析随机不精确的增强拉格朗加法的方法(Stoc-iALM),以解决涉及非混凝土复合(即光+nonsmooth)客观和不混凝土功能限制的问题。 我们采用了标准的 iALM 框架,并设计了一个子例程。 我们采用了基于动力的基于动力的变异性准随机梯度方法( Pastorm) 和后处理步骤。 在某些常规条件下( 也是在现有工程中假设的), 要达到预期的美元- KKT 点, 我们设置了一个与美元( varepslon) 目标和不平稳功能限制( 5} ) 有关的复杂性结果。 我们采用了最著名的 $O (\ varepsilon ⁇ 6} 并设计了一个子路路。 在公平性受限制的精度限制的精度方法上进行实验, 以目前最复杂的方法展示了我们目前的最佳方法的结果。