The prevalence and success of Deep Neural Network (DNN) applications in recent years have motivated research on DNN compression, such as pruning and quantization. These techniques accelerate model inference, reduce power consumption, and reduce the size and complexity of the hardware necessary to run DNNs, all with little to no loss in accuracy. However, since DNNs are vulnerable to adversarial inputs, it is important to consider the relationship between compression and adversarial robustness. In this work, we investigate the adversarial robustness of models produced by several irregular pruning schemes and by 8-bit quantization. Additionally, while conventional pruning removes the least important parameters in a DNN, we investigate the effect of an unconventional pruning method: removing the most important model parameters based on the gradient on adversarial inputs. We call this method Greedy Adversarial Pruning (GAP) and we find that this pruning method results in models that are resistant to transfer attacks from their uncompressed counterparts.
翻译:近年来深神经网络(DNN)应用的普及和成功激发了对DNN压缩(如裁剪和量化)的研究。这些技术加速了模型推断,降低了电耗,并降低了运行DNN(DNN)所需的硬件的大小和复杂性,但都很少或没有损失准确性。然而,由于DNN很容易受到对抗性投入的影响,因此必须考虑压缩和对抗性强力之间的关系。在这项工作中,我们调查了由若干非正常的裁剪计划和8位位四分法产生的模型的对抗性强强。此外,在常规裁剪除DNNN(DN)中最不重要的参数的同时,我们调查了非常规裁剪裁方法的效果:删除基于对立投入梯度的最重要模型参数。我们称之为Greedy Adversariar Prurning(GAP),我们发现,这种裁剪裁方法的结果是,模型不会将攻击从未受压迫的对应方转移。