Differentiable architecture search (DAS) has made great progress in searching for high-performance architectures with reduced computational cost. However, DAS-based methods mainly focus on searching for a repeatable cell structure, which is then stacked sequentially in multiple stages to form the networks. This configuration significantly reduces the search space, and ignores the importance of connections between the cells. To overcome this limitation, in this paper, we propose a Hierarchical Differentiable Architecture Search (H-DAS) that performs architecture search both at the cell level and at the stage level. Specifically, the cell-level search space is relaxed so that the networks can learn stage-specific cell structures. For the stage-level search, we systematically study the architectures of stages, including the number of cells in each stage and the connections between the cells. Based on insightful observations, we design several search rules and losses, and mange to search for better stage-level architectures. Such hierarchical search space greatly improves the performance of the networks without introducing expensive search cost. Extensive experiments on CIFAR10 and ImageNet demonstrate the effectiveness of the proposed H-DAS. Moreover, the searched stage-level architectures can be combined with the cell structures searched by existing DAS methods to further boost the performance. Code is available at: https://github.com/MalongTech/research-HDAS
翻译:不同的建筑搜索(DAS)在寻找高性能建筑方面取得了很大进展,计算成本降低。然而,基于DAS的方法主要侧重于寻找可重复的细胞结构,然后按多个阶段相叠以形成网络。这种配置大大减少了搜索空间,忽视了各细胞之间连接的重要性。为了克服这一局限性,我们在本文件中提议建立一个等级差异式建筑搜索(H-DAS),在单元格一级和阶段一级进行结构搜索。具体地说,细胞搜索空间松散,以便网络能够学习特定阶段的细胞结构。为了进行阶段搜索,我们系统地研究各个阶段的结构,包括每个阶段的细胞数量和各个细胞之间的联系。根据深刻的观察,我们设计了若干搜索规则和损失,以及搜索更高级建筑的曼杰。这种等级式搜索空间在不引入昂贵的搜索费用的情况下大大改善了网络的运行情况。 CIRAR10 和图像网络的广泛实验可以显示拟议的H-DAS/图像网络的有效性。为了进行阶段搜索,我们系统系统系统系统系统系统化的系统化结构,包括每个阶段的细胞数量和各个细胞之间的连接。此外,搜索系统搜索系统化的系统化结构是现有的搜索系统级/搜索系统级结构。