Reliable evaluation of adversarial defenses is a challenging task, currently limited to an expert who manually crafts attacks that exploit the defense's inner workings or approaches based on an ensemble of fixed attacks, none of which may be effective for the specific defense at hand. Our key observation is that adaptive attacks are composed of reusable building blocks that can be formalized in a search space and used to automatically discover attacks for unknown defenses. We evaluated our approach on 24 adversarial defenses and show that it outperforms AutoAttack, the current state-of-the-art tool for reliable evaluation of adversarial defenses: our tool discovered significantly stronger attacks by producing 3.0\%-50.8\% additional adversarial examples for 10 models, while obtaining attacks with slightly stronger or similar strength for the remaining models.
翻译:对对抗性防御的可靠评估是一项具有挑战性的任务,目前仅限于一名专家,他手动进行攻击,利用国防内部的操作或基于固定攻击组合的办法,这些攻击或办法对手头的具体防御可能没有效果。我们的主要看法是,适应性攻击是由在搜索空间可以正规化的可重复使用的构件组成,用来自动发现对未知防御的攻击。我们评估了24次对抗性防御的方法,并表明它优于AutoAttack(AutoAttack)(AutoAttack)(AutoAttack)(AutoAttack)(AutoAttack),这是目前可靠评估对抗性防御的最新工具:我们的工具通过为10种模式制作3.0 ⁇ -50.8++++额外对抗性攻击实例,同时为其余的模型获取较强或类似的攻击力略强或相似的攻击,从而发现攻击力大得多。