One major goal of the AI security community is to securely and reliably produce and deploy deep learning models for real-world applications. To this end, data poisoning based backdoor attacks on deep neural networks (DNNs) in the production stage (or training stage) and corresponding defenses are extensively explored in recent years. Ironically, backdoor attacks in the deployment stage, which can often happen in unprofessional users' devices and are thus arguably far more threatening in real-world scenarios, draw much less attention of the community. We attribute this imbalance of vigilance to the weak practicality of existing deployment-stage backdoor attack algorithms and the insufficiency of real-world attack demonstrations. To fill the blank, in this work, we study the realistic threat of deployment-stage backdoor attacks on DNNs. We base our study on a commonly used deployment-stage attack paradigm -- adversarial weight attack, where adversaries selectively modify model weights to embed backdoor into deployed DNNs. To approach realistic practicality, we propose the first gray-box and physically realizable weights attack algorithm for backdoor injection, namely subnet replacement attack (SRA), which only requires architecture information of the victim model and can support physical triggers in the real world. Extensive experimental simulations and system-level real-world attack demonstrations are conducted. Our results not only suggest the effectiveness and practicality of the proposed attack algorithm, but also reveal the practical risk of a novel type of computer virus that may widely spread and stealthily inject backdoor into DNN models in user devices. By our study, we call for more attention to the vulnerability of DNNs in the deployment stage.
翻译:AI安全界的一个主要目标是安全可靠地制作和部署用于现实世界应用的深层次学习模型。为此,近年来广泛探讨了基于数据中毒的数据中毒对生产阶段(或培训阶段)深神经网络的幕后攻击和相应的防御手段。具有讽刺意味的是,部署阶段的幕后攻击往往发生在不专业用户的装置中,因此在现实世界情景中可能威胁更大。我们把这种警惕不平衡归因于现有部署阶段幕后攻击算法的实用性弱和真实世界攻击演示的不足。为了填补空白,我们在这项工作中研究了部署阶段幕后攻击的现实威胁和相应的防御手段。我们的研究基于一种常用的部署阶段攻击模式 -- -- 对抗性重量攻击,对手有选择地修改模型重量以将后门嵌入部署的DNNNNW系统。为了现实现实现实现实的实用性,我们建议将第一个灰色箱和实际可实现的背后攻击算法用于后注射,即次网络替换攻击(SRA),在实际网络攻击演示阶段,我们研究部署阶段的部署阶段的实际性威胁威胁现实性威胁威胁。我们只要求真实的模型和真实的D型受害者模型,在实际世界的模型中进行中进行。我们实际的模型的部署的系统可能支持真实的系统,在实际的模型中进行中,而只是模拟的系统,而只是模拟的模拟的模拟的系统,在实际的模型的模型的模型的模型的模型的模型和真实的模型的模型。