Composite minimization is a powerful framework in large-scale convex optimization, based on decoupling of the objective function into terms with structurally different properties and allowing for more flexible algorithmic design. We introduce a new algorithmic framework for complementary composite minimization, where the objective function decouples into a (weakly) smooth and a uniformly convex term. This particular form of decoupling is pervasive in statistics and machine learning, due to its link to regularization. The main contributions of our work are summarized as follows. First, we introduce the problem of complementary composite minimization in general normed spaces; second, we provide a unified accelerated algorithmic framework to address broad classes of complementary composite minimization problems; and third, we prove that the algorithms resulting from our framework are near-optimal in most of the standard optimization settings. Additionally, we show that our algorithmic framework can be used to address the problem of making the gradients small in general normed spaces. As a concrete example, we obtain a nearly-optimal method for the standard $\ell_1$ setup (small gradients in the $\ell_{\infty}$ norm), essentially matching the bound of Nesterov (2012) that was previously known only for the Euclidean setup. Finally, we show that our composite methods are broadly applicable to a number of regression and other classes of optimization problems, where regularization plays a key role. Our methods lead to complexity bounds that are either new or match the best existing ones.
翻译:混凝土最小化是大规模 convex 优化的强大框架, 其基础是将目标功能与结构上的不同属性脱钩, 并允许更灵活的算法设计。 我们为互补的复合最小化引入一个新的算法框架, 目标函数分离为( 弱化) 顺利和一致的 convex 术语。 这种特殊的脱钩形式在统计和机器学习中十分普遍, 因为它与正规化的联系。 我们工作的主要贡献概括如下。 首先, 我们引入了在一般规范空间中互补复合复合最小化的混合优化问题; 第二, 我们提供了一个统一的加速算法框架, 以解决各种互为补充的复合最小化问题; 第三, 我们证明我们框架产生的算法在大多数标准优化设置情况下接近最佳。 此外, 我们的算法框架可以用来解决在一般规范空间中缩小梯度的问题。 具体地说, 我们获得了一种接近于标准的 $\ell_ ell_ 1 $ 1美元设置的匹配率( 小梯度在 $\\\ intimfty clas rude) 。 rudeal remailate the the the rual ruild ruil us fal ruild ruild rulest the werum rublest the wermalst thermalst rmals, rmaislviewst lexolfrviolviolviold laphtald led leds) laddd rod lad rviold rviold ladddd rmad rud rops. rupd_ rrrrrrr r_ ladddd rrrrr r r rr r rr____ r_ r_ r_ r_ laddddddddddddddddddddddd r_ r____ r_ r_ r_ r_ r_ r_ r_ rrrrrrrrrrrrrr_ r_ r_ r_ r_ r_