For the goal of automated design of high-performance deep convolutional neural networks (CNNs), Neural Architecture Search (NAS) methodology is becoming increasingly important for both academia and industries.Due to the costly stochastic gradient descent (SGD) training of CNNs for performance evaluation, most existing NAS methods are computationally expensive for real-world deployments. To address this issue, we first introduce a new performance estimation metric, named Random-Weight Evaluation (RWE) to quantify the quality of CNNs in a cost-efficient manner. Instead of fully training the entire CNN, the RWE only trains its last layer and leaves the remainders with randomly initialized weights, which results in a single network evaluation in seconds.Second, a complexity metric is adopted for multi-objective NAS to balance the model size and performance. Overall, our proposed method obtains a set of efficient models with state-of-the-art performance in two real-world search spaces. Then the results obtained on the CIFAR-10 dataset are transferred to the ImageNet dataset to validate the practicality of the proposed algorithm. Moreover, ablation studies on NAS-Bench-301 datasets reveal the effectiveness of the proposed RWE in estimating the performance compared with existing methods.
翻译:为了实现高性能深层神经神经网络(CNNs)的自动化设计目标,神经结构搜索(NAS)方法对学术界和行业都越来越重要。 由于对有线电视新闻网进行高成本的随机梯度梯度下降(SGD)培训以进行业绩评价,大多数现有的NAS方法计算起来对于现实世界的部署来说成本很高。为了解决这个问题,我们首先采用新的性能估计指标,名为随机光学评估(RWE),以具有成本效益的方式量化CNN的质量。RWE没有充分培训整个CNN,而只培训最后一层,剩下的部分则随机初始加权,结果在几秒钟内进行单一的网络评价。第二,对多目标的NAS采用了复杂度指标,以平衡模型的规模和性能。总体而言,我们拟议的方法在两个现实世界的搜索空间中获得了一套具有最新性能的模型。然后,CIRA-10数据集所取得的结果被转移到图像网络数据集,以验证拟议的算法的实用性。此外,在对NAS-BANS-30S-30S 1的拟议数据进行对比研究中,对NAS-WE-30S-30S-S-S-S-S-30S-S-S-S-S-Set的当前数据进行对比。