Different from other deep scalable architecture-based NAS approaches, Broad Neural Architecture Search (BNAS) proposes a broad scalable architecture which consists of convolution and enhancement blocks, dubbed Broad Convolutional Neural Network (BCNN), as the search space for amazing efficiency improvement. BCNN reuses the topologies of cells in the convolution block so that BNAS can employ few cells for efficient search. Moreover, multi-scale feature fusion and knowledge embedding are proposed to improve the performance of BCNN with shallow topology. However, BNAS suffers some drawbacks: 1) insufficient representation diversity for feature fusion and enhancement and 2) time consumption of knowledge embedding design by human experts. This paper proposes Stacked BNAS, whose search space is a developed broad scalable architecture named Stacked BCNN, with better performance than BNAS. On the one hand, Stacked BCNN treats mini BCNN as a basic block to preserve comprehensive representation and deliver powerful feature extraction ability. For multi-scale feature enhancement, each mini BCNN feeds the outputs of deep and broad cells to the enhancement cell. For multi-scale feature fusion, each mini BCNN feeds the outputs of deep, broad and enhancement cells to the output node. On the other hand, Knowledge Embedding Search (KES) is proposed to learn appropriate knowledge embeddings in a differentiable way. Moreover, the basic unit of KES is an over-parameterized knowledge embedding module that consists of all possible candidate knowledge embeddings. Experimental results show that 1) Stacked BNAS obtains better performance than BNAS-v2 on both CIFAR-10 and ImageNet, 2) the proposed KES algorithm contributes to reducing the parameters of the learned architecture with satisfactory performance, and 3) Stacked BNAS delivers a state-of-the-art efficiency of 0.02 GPU days.
翻译:宽度神经架构搜索(BNAS) 提出一个宽度可扩展架构,由混凝土和增强区块组成,称为宽度革命神经网络(BNNN),作为惊人效率改进的搜索空间。 BNNN 重新使用混凝土区单元格的表层结构,以便BNAS能够为高效搜索使用几个细胞。此外,还提议采用多级特征聚合和知识嵌入,用浅层表层改善 BNNNN 的性能。然而,BNAS 也有一些缺陷:(1) 特性融合和增强的0NA 代表多样性不足,以及(2) 人类专家对知识嵌入的刻度设计进行时间消耗。 本文提议Staced BNAS, 其搜索空间是一个叫做StackNCNNNN,其性能优于BS。 StackNNNC 将小型的功能整合和强大的特征提取能力作为基础部分。对于多级的模型来说,每个小型的BNNNCER将更深层和宽宽度的机基级的输出都显示高层次的功能。