Population-based search has recently emerged as a possible alternative to Reinforcement Learning (RL) for black-box neural architecture search (NAS). It performs well in practice even though it is not theoretically well understood. In particular, whereas traditional population-based search methods such as evolutionary algorithms (EAs) draw much power from crossover operations, it is difficult to take advantage of them in NAS. The main obstacle is believed to be the permutation problem: The mapping between genotype and phenotype in traditional graph representations is many-to-one, leading to a disruptive effect of standard crossover. This paper presents the first theoretical analysis of the behaviors of mutation, crossover and RL in black-box NAS, and proposes a new crossover operator based on the shortest edit path (SEP) in graph space. The SEP crossover is shown theoretically to overcome the permutation problem, and as a result, have a better expected improvement compared to mutation, standard crossover and RL. Further, it empirically outperform these other methods on state-of-the-art NAS benchmarks. The SEP crossover therefore allows taking full advantage of population-based search in NAS, and the underlying theory can serve as a foundation for deeper understanding of black-box NAS methods in general.
翻译:以人口为基础的搜索是黑箱神经结构搜索(NAS)中强化学习(RL)的一种可能的替代方法,最近作为黑箱神经结构搜索(NAS)中黑箱神经结构搜索(NAS)的一种可能的替代方法出现。它在实践中表现良好,尽管在理论上并没有得到很好的理解。特别是,传统的基于人口的搜索方法,如进化算法(EAs),从交叉操作中获得了很大的权力,但在NAS中很难利用这些方法。主要障碍据信是变异问题:传统图形表达方式中的基因类型和phone类型之间的绘图是多方面的,导致标准的交叉转换产生干扰效应。本文对黑箱NAS的突变、交叉和RL行为进行了首次理论分析,并提议在图形空间中以最短的编辑路径为基础,建立一个新的交叉操作器。SEPO交叉操作器在理论上表明要克服变异问题,结果比变异、标准交叉和RL有更好的预期改进。此外,它从经验上超越了这些其他方法,从而产生了标准的交叉影响效果。本文对黑箱中的变异、交叉和RLS基准进行首次理论分析,因此,S系统可以充分利用NAP系统的基础基础基础基础的搜索,从而充分利用了NAS的基础基础基础,从而充分利用了NAS的基础基础,从而充分利用了NAS的基础,从而可以充分利用了整个系统的基础。