Generally, residual connections are indispensable network components in building CNNs and Transformers for various downstream tasks in CV and VL, which encourages skip shortcuts between network blocks. However, the layer-by-layer loopback residual connections may also hurt the model's robustness by allowing unsuspecting input. In this paper, we proposed a simple yet strong backdoor attack method - BadRes, where the residual connections play as a turnstile to be deterministic on clean inputs while unpredictable on poisoned ones. We have performed empirical evaluations on four datasets with ViT and BEiT models, and the BadRes achieves 97% attack success rate while receiving zero performance degradation on clean data. Moreover, we analyze BadRes with state-of-the-art defense methods and reveal the fundamental weakness lying in residual connections.
翻译:一般而言,剩余连接是建立CNN和变压器以完成CV和VL中各种下游任务的不可或缺的网络组成部分,这鼓励在网络区块之间跳过捷径。然而,层层间回回回路剩余连接也可能会通过允许不怀疑输入而损害模型的稳健性。 在本文中,我们提出了一个简单而有力的后门攻击方法 — — BadRes, 剩余连接作为确定清洁投入的转动工具,同时对有毒投入的不可预测。我们用VT和BeiT模型对四个数据集进行了经验性评估,BadRes实现了97%的攻击成功率,同时对清洁数据进行了零性能退化。此外,我们用最先进的防御方法分析了BadRes,并揭示了剩余连接中存在的基本弱点。