This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before, and solved using first-order stochastic FW algorithms by applying homotopy and Nesterov's smoothing techniques to the indicator function. This work improves upon these existing results and puts forth momentum-based first-order methods that yield improved convergence rates, at par with the best known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem.
翻译:本文审议了具有两套制约因素的细微孔心优化问题:(a) 优化变数领域难以预测的确定性制约因素;和(b) 接受有效预测的确定性或随机性制约因素;这种形式的问题经常出现在半无限期方案拟定过程中,以及各种NP-硬性问题通过半无限期放松解决时。由于对第一组制约因素的预测很困难,有必要探索无预测的算法,如标准化的Frank-Wolfe(FW)算法。另一方面,第二组制约因素无法以同样的方式处理,必须作为目标功能中的一个指标功能纳入,从而使FW方法的应用复杂化。以前曾研究过类似的问题,并且通过对指标功能应用同质和Nestov的平滑技术来解决了第一组问题。这项工作改进了现有的结果,并提出了能提高相关趋同率的基于动力的一级方法。 另外,在最已知的递增的递增性矩阵应用率方面,也未制定任何已知的累变式。