Filter pruning method introduces structural sparsity by removing selected filters and is thus particularly effective for reducing complexity. Previous works empirically prune networks from the point of view that filter with smaller norm contributes less to the final results. However, such criteria has been proven sensitive to the distribution of filters, and the accuracy may hard to recover since the capacity gap is fixed once pruned. In this paper, we propose a novel filter pruning method called Asymptotic Soft Cluster Pruning (ASCP), to identify the redundancy of network based on the similarity of filters. Each filter from over-parameterized network is first distinguished by clustering, and then reconstructed to manually introduce redundancy into it. Several guidelines of clustering are proposed to better preserve feature extraction ability. After reconstruction, filters are allowed to be updated to eliminate the effect caused by mistakenly selected. Besides, various decaying strategies of the pruning rate are adopted to stabilize the pruning process and improve the final performance as well. By gradually generating more identical filters within each cluster, ASCP can remove them through channel addition operation with almost no accuracy drop. Extensive experiments on CIFAR-10 and ImageNet datasets show that our method can achieve competitive results compared with many state-of-the-art algorithms.
翻译:过滤过滤器处理方法通过删除选中的过滤器而引入结构宽度,因此对于降低复杂性特别有效。 以往的工作经验性地将网络从观点上缩小, 使用较小规范的过滤器对最终结果的贡献较少。 但是, 事实证明, 这些标准对过滤器的分布十分敏感, 并且由于能力差距一旦缩小了, 准确性可能难以恢复。 在本文件中, 我们提议了一种叫做“ Asymptatistic Soft Croft Group Prutning (ASCP) ” 的新过滤器处理方法, 以根据过滤器的相似性来确定网络的冗余。 从过度计量的网络中, 每一个过滤器首先通过集群区分, 然后重建为手工引入冗余。 提议了若干组集准则, 以更好地保存特性提取能力。 重建后, 允许过滤器更新以消除错误选择的容量差所造成的效果。 此外, 我们采用了各种运行速度衰减的战略来稳定运行过程, 并改进最后性能。 通过在每组内逐渐生成更相似的过滤器, ASCP 可以通过频道添加操作, 几乎没有精确性下降, 。 在 CIFAR- 10 和图像网络上进行广泛的实验, 将显示我们的竞争性 。