This paper introduces a novel algorithm, the Perturbed Proximal Preconditioned SPIDER algorithm (3P-SPIDER), designed to solve finite sum non-convex composite optimization. It is a stochastic Variable Metric Forward-Backward algorithm, which allows approximate preconditioned forward operator and uses a variable metric proximity operator as the backward operator; it also proposes a mini-batch strategy with variance reduction to address the finite sum setting. We show that 3P-SPIDER extends some Stochastic preconditioned Gradient Descent-based algorithms and some Incremental Expectation Maximization algorithms to composite optimization and to the case the forward operator can not be computed in closed form. We also provide an explicit control of convergence in expectation of 3P-SPIDER, and study its complexity in order to satisfy the epsilon-approximate stationary condition. Our results are the first to combine the composite non-convex optimization setting, a variance reduction technique to tackle the finite sum setting by using a minibatch strategy and, to allow deterministic or random approximations of the preconditioned forward operator. Finally, through an application to inference in a logistic regression model with random effects, we numerically compare 3P-SPIDER to other stochastic forward-backward algorithms and discuss the role of some design parameters of 3P-SPIDER.
翻译:本文介绍了一种新型算法,即用于解决有限和非convex复合优化的Pertured Proxima Procial Pretical Protical Protical Squal (PIDR) 算法(3P-PIDR),该算法是用于解决有限和定额问题的一种新型算法,该算法是用于解决非cive Prox Proximed Protical Pretical Protical (3P-PIDR) 的,旨在解决非crecial Proticle Proticle Proformation 运算法(3P-PIDR) 的有限性非cregal-production 。我们的成果是第一个结合非creducal Express report 的复合非cregrestical 设置, 一种减少差异技术,通过使用迷性战略解决限定的前端操作者的确定性或随机近似近似近似近似性算法。最后,我们通过将第3号算法的后推算法对前端分析后推算法的其他三号的后推算法,我们讨论了第3号的后推后推算法,我们的成果是第一个后推算法。