In this paper we propose a novel network adaption method called Differentiable Network Adaption (DNA), which can adapt an existing network to a specific computation budget by adjusting the width and depth in a differentiable manner. The gradient-based optimization allows DNA to achieve an automatic optimization of width and depth rather than previous heuristic methods that heavily rely on human priors. Moreover, we propose a new elastic search space that can flexibly condense or expand during the optimization process, allowing the network optimization of width and depth in a bi-direction manner. By DNA, we successfully achieve network architecture optimization by condensing and expanding in both width and depth dimensions. Extensive experiments on ImageNet demonstrate that DNA can adapt the existing network to meet different targeted computation requirements with better performance than previous methods. What's more, DNA can further improve the performance of high-accuracy networks obtained by state-of-the-art neural architecture search methods such as EfficientNet and MobileNet-v3.
翻译:在本文中,我们提出了一个新的网络适应适应方法,名为“差异网络适应”(DNA),它可以通过以不同的方式调整宽度和深度,使现有网络适应特定的计算预算。基于梯度的优化使DNA能够实现宽度和深度的自动优化,而不是严重依赖人类前科的先前的超常方法。此外,我们提出了一个新的弹性搜索空间,可以在优化过程中灵活压缩或扩展,允许以双向方式优化网络宽度和深度。通过DNA,我们成功地实现了网络结构优化,在宽度和深度两个维度上都进行了凝聚和扩大。图像网络的广泛实验表明,DNA能够比以往方法更好地调整现有网络,以适应不同目标的计算要求。此外,DNA还可以进一步改进通过高效网络和移动网络-V3等最先进的神经结构搜索方法获得的高准确性网络的性能。