Recent advances in convolutional neural networks(CNNs) usually come with the expense of excessive computational overhead and memory footprint. Network compression aims to alleviate this issue by training compact models with comparable performance. However, existing compression techniques either entail dedicated expert design or compromise with a moderate performance drop. In this paper, we propose a novel structured sparsification method for efficient network compression. The proposed method automatically induces structured sparsity on the convolutional weights, thereby facilitating the implementation of the compressed model with the highly-optimized group convolution. We further address the problem of inter-group communication with a learnable channel shuffle mechanism. The proposed approach can be easily applied to compress many network architectures with a negligible performance drop. Extensive experimental results and analysis demonstrate that our approach gives a competitive performance against the recent network compression counterparts with a sound accuracy-complexity trade-off.
翻译:网络压缩的目的是通过培训具有类似性能的紧凑模型来缓解这一问题。然而,现有的压缩技术要么需要专家专门设计,要么需要以中等性能下降来妥协。在本文中,我们建议为高效网络压缩而采用新的结构化封闭方法。拟议方法自动导致在累进权重上结构化的宽度,从而便利在高度优化的群落中实施压缩模型。我们进一步解决了以可学习的频道打乱机制进行群体间沟通的问题。拟议方法可以很容易地用于压缩许多网络结构,但性能下降微不足道。广泛的实验结果和分析表明,我们的方法与最近的网络压缩对应方相比具有竞争性,具有健全的准确性-兼容性交易。