Zeroth-order optimization methods are developed to overcome the practical hurdle of having knowledge of explicit derivatives. Instead, these schemes work with merely access to noisy functions evaluations. The predominant approach is to mimic first-order methods by means of some gradient estimator. The theoretical limitations are well-understood, yet, as most of these methods rely on finite-differencing for shrinking differences, numerical cancellation can be catastrophic. The numerical community developed an efficient method to overcome this by passing to the complex domain. This approach has been recently adopted by the optimization community and in this work we analyze the practically relevant setting of dealing with computational noise. To exemplify the possibilities we focus on the strongly-convex optimization setting and provide a variety of non-asymptotic results, corroborated by numerical experiments, and end with local non-convex optimization.
翻译:零顺序优化方法是为了克服了解明显衍生物的实际障碍而开发的。相反,这些计划只是利用噪音功能评估。主要的方法是通过一些梯度估计器模仿一阶方法。理论限制是广为人知的,然而,由于大多数这些方法依靠有限的差异差异来缩小差异,数字取消可能是灾难性的。数字群体制定了一种有效的方法,通过转而到复杂的领域来克服这一点。这个方法最近被优化社区所采用,在这项工作中,我们分析了处理计算噪音的实际相关环境。举例说明了我们侧重于强电流优化设置的可能性,并提供了各种非零位结果,这些结果得到了数字实验的证实,并以当地非电流优化为目的。