We study statistical estimators computed using iterative optimization methods that are not run until completion. Classical results on maximum likelihood estimators (MLEs) assert that a one-step estimator (OSE), in which a single Newton-Raphson iteration is performed from a starting point with certain properties, is asymptotically equivalent to the MLE. We further develop these early-stopping results by deriving properties of one-step estimators defined by a single iteration of scaled proximal methods. Our main results show the asymptotic equivalence of the likelihood-based estimator and various one-step estimators defined by scaled proximal methods. By interpreting OSEs as the last of a sequence of iterates, our results provide insight on scaling numerical tolerance with sample size. Our setting contains scaled proximal gradient descent applied to certain composite models as a special case, making our results applicable to many problems of practical interest. Additionally, our results provide support for the utility of the scaled Moreau envelope as a statistical smoother by interpreting scaled proximal descent as a quasi-Newton method applied to the scaled Moreau envelope.
翻译:我们用不至完成的迭代优化方法计算出统计估计值。关于最大可能性估计值(MLEs)的经典结果显示,单步估计值(OSE),即单步估计值(OSE),从某一起点对某些属性进行单牛顿-拉弗森的迭代,与MLE基本相同。我们进一步通过得出由按比例缩放的准度方法的一次迭代定义的一步估计值的属性来发展这些早期结束结果。我们的主要结果显示,根据按比例缩放的准度方法定义的概率估计值和各种一步估计值的零步估计值(OSE),通过将单步估算值解释为某一序列的最后一个迭代序列,我们的结果可以洞察到与样本大小的数值容忍度。我们的设置包含对某些复合模型应用的按比例缩放的准度梯度梯度梯度梯度的特性,使我们的结果适用于许多实际感兴趣的问题。此外,我们的主要结果提供了对以缩放波域为统计平衡的效用的支持,通过将缩缩缩缩成一个统计平稳的模型,将缩缩缩缩成一个准纽,将缩成一个缩成一个缩缩缩缩缩成一个缩成一个准新纽。