The computational demands of neural architecture search (NAS) algorithms are usually directly proportional to the size of their target search spaces. Thus, limiting the search to high-quality subsets can greatly reduce the computational load of NAS algorithms. In this paper, we present Clustering-Based REDuction (C-BRED), a new technique to reduce the size of NAS search spaces. C-BRED reduces a NAS space by clustering the computational graphs associated with its architectures and selecting the most promising cluster using proxy statistics correlated with network accuracy. When considering the NAS-Bench-201 (NB201) data set and the CIFAR-100 task, C-BRED selects a subset with 70% average accuracy instead of the whole space's 64% average accuracy.
翻译:神经结构搜索算法(NAS)的计算要求通常与其目标搜索空间的大小直接成正比。 因此, 将搜索限制在高质量的子集中可以大大减少NAS算法的计算负荷。 在本文中, 我们介绍了一种缩小NAS搜索空间规模的新技术( C- BRED ) 。 C- BRED 将与其结构相关的计算图组合在一起, 并使用与网络准确性相关的代理统计数据选择最有前景的组群, 从而减少NAS 空间。 在考虑NAS- Bench-201(NB201)数据集和CIFAR- 100任务时, C- BRED 选择了一个平均精度为70%的子集, 而不是整个空间平均精度为64%的子。