In the paper, we propose a class of faster adaptive Gradient Descent Ascent (GDA) methods for solving the nonconvex-strongly-concave minimax problems based on unified adaptive matrices, which include almost existing coordinate-wise and global adaptive learning rates. Specifically, we propose a fast Adaptive Gradient Decent Ascent (AdaGDA) method based on the basic momentum technique, which reaches a lower gradient complexity of $O(\kappa^4\epsilon^{-4})$ for finding an $\epsilon$-stationary point without large batches, which improves the results of the existing adaptive GDA methods by a factor of $O(\sqrt{\kappa})$. At the same time, we present an accelerated version of AdaGDA (VR-AdaGDA) method based on the momentum-based variance reduced technique, which achieves a lower gradient complexity of $O(\kappa^{4.5}\epsilon^{-3})$ for finding an $\epsilon$-stationary point without large batches, which improves the results of the existing adaptive GDA methods by a factor of $O(\epsilon^{-1})$. Moreover, we prove that our VR-AdaGDA method reaches the best known gradient complexity of $O(\kappa^{3}\epsilon^{-3})$ with the mini-batch size $O(\kappa^3)$. In particular, we provide an effective convergence analysis framework for our adaptive GDA methods. Some experimental results on policy evaluation and fair classifier tasks demonstrate the efficiency of our algorithms.
翻译:在论文中,我们建议了一种基于统一适应矩阵(包括几乎现有的协调型和全球适应型学习率)解决非稳定型小型数学问题的快速适应性梯度梯度梯度梯度方法。具体地说,我们建议了一种基于基本动力技术的快速适应性梯度梯度梯度标准(AdaGDA)方法(AdaGDA),其梯度复杂性为O(kappa)4\eepsilon ⁇ 4}(GDA)美元,以找到一个没有大批量的固定点(eepsilon$),这通过一个系数改善现有适应性GDA方法(V-AGDA)的收效。