We study the problem of finding a near-stationary point for smooth minimax optimization. The recent proposed extra anchored gradient (EAG) methods achieve the optimal convergence rate for the convex-concave minimax problem in deterministic setting. However, the direct extension of EAG to stochastic optimization is not efficient.In this paper, we design a novel stochastic algorithm called Recursive Anchored IteratioN (RAIN). We show that the RAIN achieves near-optimal stochastic first-order oracle (SFO) complexity for stochastic minimax optimization in both convex-concave and strongly-convex-strongly-concave cases. In addition, we extend the idea of RAIN to solve structured nonconvex-nonconcave minimax problem and it also achieves near-optimal SFO complexity.
翻译:我们研究的是寻找接近静止的点的问题,以便顺利地优化微型通量。最近提出的额外锁定梯度方法(EAG)在确定性环境下实现了 convex-concave 微型通量问题的最佳趋同率。然而,EAG直接延伸至随机优化并不有效。在本文中,我们设计了一种新型的随机算法,称为Recursiive Ancrocred IteratioN(RAIN)。我们表明,REAIN在 convex-concave 和强凝固混凝固的案例中,都实现了对合成微型通量微型通量优化的近最佳先令(SFO)复杂度。此外,我们扩大了REA的构想,以解决结构化的非convex-nonconcolave 小型通量问题,并实现了接近最佳的SFO复杂度。