The efficient, automated search for well-performing neural architectures (NAS) has drawn increasing attention in the recent past. Thereby, the predominant research objective is to reduce the necessity of costly evaluations of neural architectures while efficiently exploring large search spaces. To this aim, surrogate models embed architectures in a latent space and predict their performance, while generative models for neural architectures enable optimization-based search within the latent space the generator draws from. Both, surrogate and generative models, have the aim of facilitating query-efficient search in a well-structured latent space. In this paper, we further improve the trade-off between query-efficiency and promising architecture generation by leveraging advantages from both, efficient surrogate models and generative design. To this end, we propose a generative model, paired with a surrogate predictor, that iteratively learns to generate samples from increasingly promising latent subspaces. This approach leads to very effective and efficient architecture search, while keeping the query amount low. In addition, our approach allows in a straightforward manner to jointly optimize for multiple objectives such as accuracy and hardware latency. We show the benefit of this approach not only w.r.t. the optimization of architectures for highest classification accuracy but also in the context of hardware constraints and outperform state-of-the-art methods on several NAS benchmarks for single and multiple objectives. We also achieve state-of-the-art performance on ImageNet. The code is available at http://github.com/jovitalukasik/AG-Net .
翻译:高效、自动化的神经结构搜索(NAS)在最近的过去引起了人们越来越多的关注。 因此,主要的研究目标是减少对神经结构进行昂贵的评估的必要性,同时有效地探索大型搜索空间。为此,代用模型将结构嵌入潜伏空间并预测其性能,而神经结构的基因化模型可以在发电机所利用的潜在空间内进行基于优化的搜索。 代用模型和基因化模型都旨在便利在结构完善的潜在空间内进行高效的查询搜索。 在本文中,我们通过利用高效的代用模型和基因化设计,进一步改进对神经结构结构结构进行低成本评估的必要性。为此,我们提出了一种配以隐蔽空间的基因模型,该模型反复学习从日益充满希望的潜在子空间中生成样本。这种方法导致非常有效和高效的架构搜索,同时保持现有查询数量低。 此外,我们的方法允许AG以直接的方式联合优化多种目标,如精确度和硬性硬性硬性硬性硬性硬性硬性硬性标值/硬性标值的生成。 我们还在多种硬性能限制方面展示了这一方法的好处。