Channel pruning is used to reduce the number of weights in a Convolutional Neural Network (CNN). Channel pruning removes slices of the weight tensor so that the convolution layer remains dense. The removal of these weight slices from a single layer causes mismatching number of feature maps between layers of the network. A simple solution is to force the number of feature map between layers to match through the removal of weight slices from subsequent layers. This additional constraint becomes more apparent in DNNs with branches where multiple channels need to be pruned together to keep the network dense. Popular pruning saliency metrics do not factor in the structural dependencies that arise in DNNs with branches. We propose Domino metrics (built on existing channel saliency metrics) to reflect these structural constraints. We test Domino saliency metrics against the baseline channel saliency metrics on multiple networks with branches. Domino saliency metrics improved pruning rates in most tested networks and up to 25% in AlexNet on CIFAR-10.
翻译:频道剪切用于减少进化神经网络(CNN)的重量数量; 频道剪切去除重量的分片,使进化层保持密度; 从一个层移走这些重量切片,造成网络各层间地貌图的不匹配。 一个简单的解决办法是迫使各层间地貌图的数量通过从随后的层中去除重量切片来匹配。 在有分支的DNN中,这种额外的限制更加明显,需要多频道连接以保持网络的密度。 流行的突出度量在DNN和分支之间产生的结构依赖性中没有因素。 我们建议多米诺测量仪(在现有频道特征度量度仪上建立)以反映这些结构性限制。 我们测试多米诺在多个分支网络上的基线频道显著度量度。 多米诺明显度量测量仪改进了大多数测试网络的运行率,在CIFAR-10的AlexNet中提高了25%。