The goal of this paper is to reduce the total complexity of gradient-based methods for two classes of problems: affine-constrained composite convex optimization and bilinear saddle-point structured non-smooth convex optimization. Our technique is based on a double-loop inexact accelerated proximal gradient (APG) method for minimizing the summation of a non-smooth but proximable convex function and two smooth convex functions with different smoothness constants and computational costs. Compared to the standard APG method, the inexact APG method can reduce the total computation cost if one smooth component has higher computational cost but a smaller smoothness constant than the other. With this property, the inexact APG method can be applied to approximately solve the subproblems of a proximal augmented Lagrangian method for affine-constrained composite convex optimization and the smooth approximation for bilinear saddle-point structured non-smooth convex optimization, where the smooth function with a smaller smoothness constant has significantly higher computational cost. Thus it can reduce total complexity for finding an approximately optimal/stationary solution. This technique is similar to the gradient sliding technique in the literature. The difference is that our inexact APG method can efficiently stop the inner loop by using a computable condition based on a measure of stationarity violation, while the gradient sliding methods need to pre-specify the number of iterations for the inner loop. Numerical experiments demonstrate significantly higher efficiency of our methods over an optimal primal-dual first-order method and the gradient sliding methods.
翻译:本文的目标是降低两类问题基于梯度的方法的复杂程度: 松散的复合锥形优化和双线马鞍点结构非moos convex优化。 我们的技术基于双环不超快加速准度梯度(APG)法, 以尽量减少非单向但相近的粘结函数和两个平滑的粘结函数的相加性, 并具有不同的平滑常态常量和计算成本。 与标准 APG 方法相比, 如果一个平滑的组件有更高的内部计算成本, 且比其他更低的平滑马鞍点结构的平滑性, APG 方法可以降低总计算成本。 有了这个属性, 异常的APG方法可以用于大约解决一个非双向增强的拉格朗格方法的相加加和两个顺流的调和函数, 双线马鞍点结构非movex优化, 与较平滑的平滑函数相比, 平滑的平滑性计算成本会大大高于其他的计算成本, 但平滑的平滑性平滑性方法可以用来测量我们最优的平流的平流方法, 。 平流的平流的平流方法可以降低的平流的平流法 。