Natural Evolution Strategies (NES) is a promising framework for black-box continuous optimization problems. NES optimizes the parameters of a probability distribution based on the estimated natural gradient, and one of the key parameters affecting the performance is the learning rate. We argue that from the viewpoint of the natural gradient method, the learning rate should be determined according to the estimation accuracy of the natural gradient. To do so, we propose a new learning rate adaptation mechanism for NES. The proposed mechanism makes it possible to set a high learning rate for problems that are relatively easy to optimize, which results in speeding up the search. On the other hand, in problems that are difficult to optimize (e.g., multimodal functions), the proposed mechanism makes it possible to set a conservative learning rate when the estimation accuracy of the natural gradient seems to be low, which results in the robust and stable search. The experimental evaluations on unimodal and multimodal functions demonstrate that the proposed mechanism works properly depending on a search situation and is effective over the existing method, i.e., using the fixed learning rate.
翻译:自然进化战略(NES)是解决黑盒连续优化问题的有希望的框架。国家空间局优化了基于估计自然梯度的概率分布参数,影响绩效的一个关键参数是学习率。我们认为,从自然梯度方法的角度来看,学习率应该根据自然梯度的估计准确度来确定。为此,我们提议了一个新的国家空间局学习率适应机制。拟议机制使得有可能为相对容易优化的问题设定高学习率,从而加快搜索速度。另一方面,在难以优化的问题(例如多式联运功能)中,拟议机制使得在自然梯度估计准确度似乎很低的情况下,有可能设定保守的学习率,从而形成稳健和稳定的搜索结果。对单式和多式功能的实验性评估表明,拟议的机制根据搜索情况适当运作,对现行方法有效,即使用固定学习率。