Neural architecture search is a promising area of research dedicated to automating the design of neural network models. This field is rapidly growing, with a surge of methodologies ranging from Bayesian optimization,neuroevoltion, to differentiable search, and applications in various contexts. However, despite all great advances, few studies have presented insights on the difficulty of the problem itself, thus the success (or fail) of these methodologies remains unexplained. In this sense, the field of optimization has developed methods that highlight key aspects to describe optimization problems. The fitness landscape analysis stands out when it comes to characterize reliably and quantitatively search algorithms. In this paper, we propose to use fitness landscape analysis to study a neural architecture search problem. Particularly, we introduce the fitness landscape footprint, an aggregation of eight (8)general-purpose metrics to synthesize the landscape of an architecture search problem. We studied two problems, the classical image classification benchmark CIFAR-10, and the Remote-Sensing problem So2Sat LCZ42. The results present a quantitative appraisal of the problems, allowing to characterize the relative difficulty and other characteristics, such as the ruggedness or the persistence, that helps to tailor a search strategy to the problem. Also, the footprint is a tool that enables the comparison of multiple problems.
翻译:神经结构搜索是专门致力于神经网络模型设计自动化的一个有希望的研究领域,是一个致力于神经网络模型设计自动化的有希望的研究领域。这个领域正在迅速发展,从巴伊西亚优化、神经蒸发、神经结构搜索、不同搜索和应用等方法激增到不同背景下的不同搜索和应用。然而,尽管取得了巨大的进步,很少有研究揭示了问题本身的难度,因此这些方法的成功(或失败)仍然无法解释。从这个意义上讲,优化领域开发了突出描述优化问题的关键方面的方法。健身环境分析在可靠和定量搜索算法方面表现得非常突出。在本文件中,我们提议利用健身环境分析来研究神经结构搜索问题。特别是,我们引入了健身环境足迹(8个通用指标),以综合建筑搜索问题的全貌。我们研究了两个问题:古典图像分类基准CIFAR-10和遥感问题S2Sat LCZ42。结果对问题进行了定量评估,从而可以辨别相对困难和其他特征,例如崎岖或持久性。此外,我们建议利用一个多处的比较工具来调整搜索战略的足迹。