Deep convolutional neural networks (CNNs) with a large number of parameters requires huge computational resources, which has limited the application of CNNs on resources constrained appliances. Decomposition-based methods, therefore, have been utilized to compress CNNs in recent years. However, since the compression factor and performance are negatively correlated, the state-of-the-art works either suffer from severe performance degradation or have limited low compression factors. To overcome these problems, unlike previous works compressing layers separately, we propose to compress CNNs and alleviate performance degradation via joint matrix decomposition. The idea is inspired by the fact that there are lots of repeated modules in CNNs, and by projecting weights with the same structures into the same subspace, networks can be further compressed and even accelerated. In particular, three joint matrix decomposition schemes are developed, and the corresponding optimization approaches based on Singular Values Decomposition are proposed. Extensive experiments are conducted across three challenging compact CNNs and 3 benchmark data sets to demonstrate the superior performance of our proposed algorithms. As a result, our methods can compress the size of ResNet-34 by 22x with slighter accuracy degradation compared with several state-of-the-art methods.
翻译:具有大量参数的深相神经网络(CNN)需要大量计算资源,这限制了CNN对资源受限电器的应用。因此,近年来采用了基于分解的方法压缩CNN。然而,由于压缩系数和性能呈负相关,因此,由于压缩系数和性能呈负相关,最先进的工程要么出现严重性能退化,要么存在有限的低压缩因素。为了克服这些问题,与以往分别压缩层的工程不同,我们提议压缩CNN,并通过联合矩阵分解减轻性能退化来缓解性能退化。这种想法的灵感来自有众多CNN的重复模块,以及在同一次空间投射同一结构中的重力,因此网络可以进一步压缩甚至加速。特别是,开发了三个联合矩阵分解方案,并提出了基于星体价值的相应的优化方法。我们通过三个具有挑战性的压缩式CNN和3个基准数据集进行了广泛的实验,以显示我们提议的算法的优劣性。我们的方法可以将ResNet-34的大小与若干次状态相比,将ResNet-34的精度降解方法压缩为22。