In many statistical learning problems, it is desired that the optimal solution conforms to an a priori known sparsity structure represented by a directed acyclic graph. Inducing such structures by means of convex regularizers requires nonsmooth penalty functions that exploit group overlapping. Our study focuses on evaluating the proximal operator of the Latent Overlapping Group lasso developed by Jacob et al. (2009). We implemented an Alternating Direction Method of Multiplier with a sharing scheme to solve large-scale instances of the underlying optimization problem efficiently. In the absence of strong convexity, global linear convergence of the algorithm is established using the error bound theory. More specifically, the paper contributes to establishing primal and dual error bounds when the nonsmooth component in the objective function does not have a polyhedral epigraph. We also investigate the effect of the graph structure on the speed of convergence of the algorithm. Detailed numerical simulation studies over different graph structures supporting the proposed algorithm and two applications in learning are provided.
翻译:在许多统计学习问题中,理想的解决方案是,优化的解决方案符合以定向环状图为代表的先验已知的宽度结构。通过正统正规化者引导这种结构需要非移动的处罚功能,以利用群体重叠。我们的研究侧重于评价雅各布等人(2009年)开发的Lident重叠小组的准操作者。我们采用了一个互换方向方法,采用一个共享方案,以有效解决潜在的优化问题中的大规模案例。在没有强大的交融性的情况下,算法的全球线性趋同使用错误约束理论。更具体地说,当目标函数中的非移动部分没有多位分布式成形时,该文件有助于建立原始和双重错误的界限。我们还调查了图形结构对算法趋同速度的影响。提供了支持拟议算法和两种学习应用的不同图形结构的详细数字模拟研究。