With the increasing popularity of deep learning, Convolutional Neural Networks (CNNs) have been widely applied in various domains, such as image classification and object detection, and achieve stunning success in terms of their high accuracy over the traditional statistical methods. To exploit the potential of CNN models, a huge amount of research and industry efforts have been devoted to optimizing CNNs. Among these endeavors, CNN architecture design has attracted tremendous attention because of its great potential of improving model accuracy or reducing model complexity. However, existing work either introduces repeated training overhead in the search process or lacks an interpretable metric to guide the design. To clear these hurdles, we propose Information Field (IF), an explainable and easy-to-compute metric, to estimate the quality of a CNN architecture and guide the search process of designs. To validate the effectiveness of IF, we build a static optimizer to improve the CNN architectures at both the stage level and the kernel level. Our optimizer not only provides a clear and reproducible procedure but also mitigates unnecessary training efforts in the architecture search process. Extensive experiments and studies show that the models generated by our optimizer can achieve up to 5.47% accuracy improvement and up to 65.38% parameters deduction, compared with state-of-the-art CNN structures like MobileNet and ResNet.
翻译:随着深层学习的日益普及,革命神经网络(CNNs)被广泛应用于各个领域,例如图像分类和物体探测,在传统统计方法的高度精准性方面取得了惊人的成功。为了利用CNN模型的潜力,已经投入了大量研究和工业努力来优化CNN。在这些努力中,CNN的建筑设计吸引了巨大的关注,因为它具有提高模型准确性或降低模型复杂性的巨大潜力。然而,现有的工作要么在搜索过程中反复引入了培训间接费用,要么缺乏可解释的参数来指导设计。为了清除这些障碍,我们建议信息领域(IF),这是一个可以解释的、易于计算的衡量标准,以估计CNN结构的质量,指导设计搜索过程。为了验证IFD的有效性,我们建立了一个静态的优化器来改进CNN的阶段级和内核结构。我们的优化器不仅提供了清晰和可追溯的程序,而且还减少了建筑搜索过程中不必要的培训努力。广泛的实验和研究显示,我们优化的模型可以达到5.38的网络的精确度,比RISM(IS-R)的精确度和精确度(RIS-R)的精确度,如5.47%的精确度。