Weight pruning is among the most popular approaches for compressing deep convolutional neural networks. Recent work suggests that in a randomly initialized deep neural network, there exist sparse subnetworks that achieve performance comparable to the original network. Unfortunately, finding these subnetworks involves iterative stages of training and pruning, which can be computationally expensive. We propose Structured Sparse Convolution (SSC), which leverages the inherent structure in images to reduce the parameters in the convolutional filter. This leads to improved efficiency of convolutional architectures compared to existing methods that perform pruning at initialization. We show that SSC is a generalization of commonly used layers (depthwise, groupwise and pointwise convolution) in ``efficient architectures.'' Extensive experiments on well-known CNN models and datasets show the effectiveness of the proposed method. Architectures based on SSC achieve state-of-the-art performance compared to baselines on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet classification benchmarks.
翻译:重力运行是压缩深层进化神经网络最受欢迎的方法之一。最近的工作表明,在一个随机初始的深层神经网络中,存在着能取得与原始网络相似的性能的稀薄子网络。 不幸的是,发现这些子网络涉及培训和运行的迭代阶段,这在计算上可能非常昂贵。我们提议结构松散(SSC),利用图像中固有的结构来减少进化过滤器的参数。这导致与在初始化时进行快速运行的现有方法相比,革命结构的效率有所提高。我们表明,SSC是“高效架构”中常用的层(深度、分组和点相近层)的通用化(深度、群体性和点相近性)。关于著名的CNN模型和数据集的广泛实验显示了拟议方法的有效性。基于SSC的建筑实现了与CIFAR-10、CIFAR-100、小智能-IMageNet和图像网络分类基准的状态性能。