The computational complexity of leveraging deep neural networks for extracting deep feature representations is a significant barrier to its widespread adoption, particularly for use in embedded devices. One particularly promising strategy to addressing the complexity issue is the notion of evolutionary synthesis of deep neural networks, which was demonstrated to successfully produce highly efficient deep neural networks while retaining modeling performance. Here, we further extend upon the evolutionary synthesis strategy for achieving efficient feature extraction via the introduction of a stress-induced evolutionary synthesis framework, where stress signals are imposed upon the synapses of a deep neural network during training to induce stress and steer the synthesis process towards the production of more efficient deep neural networks over successive generations and improved model fidelity at a greater efficiency. The proposed stress-induced evolutionary synthesis approach is evaluated on a variety of different deep neural network architectures (LeNet5, AlexNet, and YOLOv2) on different tasks (object classification and object detection) to synthesize efficient StressedNets over multiple generations. Experimental results demonstrate the efficacy of the proposed framework to synthesize StressedNets with significant improvement in network architecture efficiency (e.g., 40x for AlexNet and 33x for YOLOv2) and speed improvements (e.g., 5.5x inference speed-up for YOLOv2 on an Nvidia Tegra X1 mobile processor).
翻译:利用深心神经网络来提取深地地貌特征描述的计算复杂性是其广泛采用,特别是用于嵌入装置的一个重大障碍。解决复杂问题的一个特别有希望的战略是深心神经网络的进化合成概念,它证明能够成功地产生高效的深心神经网络,同时保持模型性能。这里,我们进一步扩展了通过采用压力诱发的进化合成框架实现高效地提取地貌特征的进化合成战略,在培训过程中对深心神经网络的突触施加了压力信号,以引起压力并引导合成过程,从而生产出代代间更高效的深心电网络,并以更高的效率改进模型性能。拟议的压力诱发性演化合成方法是对各种不同的深心神经网络结构(LeNet5, AlexNet, 和 YOLOv2)进行评估,目的是通过引入一种不同的任务(对象分类和对象探测)来合成多代间高效的应力网络。实验结果显示,拟议的将压力网络合成经压力网络的网络结构效率得到显著改进(例如,Alexx 40x Net 和33x AL-NOOvv2 的移动速度的X) 和网络的Xeurviax。