Network pruning reduces the size of neural networks by removing (pruning) neurons such that the performance drop is minimal. Traditional pruning approaches focus on designing metrics to quantify the usefulness of a neuron which is often quite tedious and sub-optimal. More recent approaches have instead focused on training auxiliary networks to automatically learn how useful each neuron is however, they often do not take computational limitations into account. In this work, we propose a general methodology for pruning neural networks. Our proposed methodology can prune neural networks to respect pre-defined computational budgets on arbitrary, possibly non-differentiable, functions. Furthermore, we only assume the ability to be able to evaluate these functions for different inputs, and hence they do not need to be fully specified beforehand. We achieve this by proposing a novel pruning strategy via constrained reinforcement learning algorithms. We prove the effectiveness of our approach via comparison with state-of-the-art methods on standard image classification datasets. Specifically, we reduce 83-92.90 of total parameters on various variants of VGG while achieving comparable or better performance than that of original networks. We also achieved 75.09 reduction in parameters on ResNet18 without incurring any loss in accuracy.
翻译:网络运行减少神经网络的大小,方法是消除(修剪)神经神经网络,使性能下降幅度极小。传统的修剪方法侧重于设计计量标准,以量化神经元的有用性,这种神经元往往相当乏味和亚最佳。较新的方法则侧重于培训辅助网络,以便自动了解每个神经元如何有用,但是它们往往没有考虑到计算限制。在这项工作中,我们提出了运行神经网络的一般方法。我们建议的方法可以使神经网络在任意性、可能无法区分的功能方面尊重预先确定的计算预算。此外,我们仅假设能够评估不同投入的这些功能,因此不需要事先充分说明。我们通过限制强化学习算法提出新的修剪裁战略,从而实现这一目标。我们通过在标准图像分类数据集方面与最先进的方法进行比较,证明了我们的方法的有效性。具体地说,我们减少了关于VGG的各种变量的总参数的83-92.90,同时实现可比较性或比原始网络的精确性能更好的性能。我们还在网络中实现了减少75.09。