A New Insight on Augmented Lagrangian Method and Its Extensions

Motivated by the recent work [He-Yuan, Balanced Augmented Lagrangian Method for Convex Programming, arXiv: 2108.08554v1, (2021)], a novel Augmented Lagrangian Method (ALM) has been proposed for solving a family of convex optimization problem subject to equality or inequality constraint. This new method is then extended to solve the multi-block separable convex optimization problem, and two related primal-dual hybrid gradient algorithms are also discussed. Preliminary and some new convergence results are established with the aid of variational analysis for both the saddle point of the problem and the first-order optimality conditions of involved subproblems.