Composite minimization is a powerful framework in large-scale convex optimization, based on decoupling of the objective function into terms with structurally different properties and allowing for more flexible algorithmic design. In this work, we introduce a new algorithmic framework for complementary composite minimization, where the objective function decouples into a (weakly) smooth and a uniformly convex term. This particular form of decoupling is pervasive in statistics and machine learning, due to its link to regularization. The main contributions of our work are summarized as follows. First, we introduce the problem of complementary composite minimization in general normed spaces; second, we provide a unified accelerated algorithmic framework to address broad classes of complementary composite minimization problems; and third, we prove that the algorithms resulting from our framework are near-optimal in most of the standard optimization settings. Additionally, we show that our algorithmic framework can be used to address the problem of making the gradients small in general normed spaces. As a concrete example, we obtain a nearly-optimal method for the standard $\ell_1$ setup (small gradients in the $\ell_\infty$ norm), essentially matching the bound of Nesterov (2012) that was previously known only for the Euclidean setup. Finally, we show that our composite methods are broadly applicable to a number of regression problems, leading to complexity bounds that are either new or match the best existing ones.
翻译:在大规模 convex 优化中, 混凝土的最小化是一个强大的框架, 其基础是将目标功能与结构上的不同属性脱钩, 并允许更灵活的算法设计。 在这项工作中, 我们引入了一个新的逻辑框架, 以互补的复合最小化, 将目标函数分离成一个( 微弱的) 顺畅的和一致的 convex 术语。 这种特殊的脱钩形式在统计和机器学习中十分普遍, 因为它与正规化有关。 我们工作的主要贡献概括如下。 首先, 我们引入了在一般规范空间中互补综合最小化功能的问题; 第二, 我们提供了一个统一的加速算法框架, 以解决各种互为互补的复合最小化问题; 第三, 我们证明我们框架产生的算法在大多数标准优化环境中是接近最优化的。 此外, 我们的算法框架可以用来解决在一般规范空间中使梯度小一些问题。 具体地说, 我们只获得了一种接近最优化的方法, $\ $ 1% 设置的标准( 最小的梯度在 $\ calclevlead develop develop rual rual) rual rual exup the the the we supilate roduild the we she the put the put the prealtiquest faltime legildlegild extime to the put the paltime a putd rotime a pal ledaldaldaldaldald rotime a palticildaldalddddd robild.