Deep learning has made breakthroughs and substantial in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers' prior knowledge and experience. And due to the limitations of human' inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture. Neural Architecture Search (NAS) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. Besides, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.
翻译:深层次的学习在许多领域都取得了突破和实质性的突破,这是因为它具有强大的自动代表能力。已经证明神经结构的设计对于数据和最终性能的特征表现至关重要。然而,神经结构的设计在很大程度上依赖于研究人员先前的知识和经验。由于人类固有知识的局限性,人们很难跳出最初的思维范式,设计一个最佳模式。因此,一个直觉的想法是尽可能减少人类的干预,让算法自动设计神经结构。神经结构搜索(NAS)只是这样一个革命性算法,而相关的研究工作是复杂而丰富的。因此,对神经结构结构的全面和系统调查至关重要。以前,有关调查主要依靠研究人员先前的知识和经验,开始将现有工作分类,主要根据NAS的关键组成部分:搜索空间、搜索战略和评价战略。虽然这种分类方法更直观,读者也很难理解有关的挑战和里程碑性工作。因此,在这次调查中,我们提供了一个新的视角:首先概述NAS算法最早的算法的特征,因此,对NAS系统的相关研究工作进行综合分析,最后我们提供了这些可能进行的全面研究。