When is Momentum Extragradient Optimal? A Polynomial-Based Analysis

The extragradient method has recently gained increasing attention, due to its convergence behavior on smooth games. In $n$-player differentiable games, the eigenvalues of the Jacobian of the vector field are distributed on the complex plane. Thus, compared to classical (i.e., single player) minimization, games exhibit more convoluted dynamics, where the extragradient method succeeds while simple gradient method could fail. Yet, in this work, instead of focusing on a specific problem class, we follow a reverse path: starting from the momentum extragradient method as the selected optimizer, and using polynomial-based analyses, we identify problem subclasses where the use of momentum in extragradient motions lead to optimal performance. Based on the hyperparameter setup, we show that the extragradient with momentum exhibits three different modes of convergence: when the eigenvalues are distributed $i)$ on the real line, $ii)$ both on the real line along with complex conjugates, and $iii)$ only as complex conjugates. We then derive the optimal hyperparameters for each case, and show that it achieves an accelerated convergence rate.