Adversarial training (AT) and its variants are the most effective approaches for obtaining adversarially robust models. A unique characteristic of AT is that an inner maximization problem needs to be solved repeatedly before the model weights can be updated, which makes the training slow. FGSM AT significantly improves its efficiency but it fails when the step size grows. The SOTA GradAlign makes FGSM AT compatible with a higher step size, however, its regularization on input gradient makes it 3 to 4 times slower than FGSM AT. Our proposed NoiseAug removes the extra computation overhead by directly regularizing on the input itself. The key contribution of this work lies in an empirical finding that single-step FGSM AT is not as hard as suggested in the past line of work: noise augmentation is all you need for (FGSM) fast AT. Towards understanding the success of our NoiseAug, we perform an extensive analysis and find that mitigating Catastrophic Overfitting (CO) and Robust Overfitting (RO) need different augmentations. Instead of more samples caused by data augmentation, we identify what makes NoiseAug effective for preventing CO might lie in its improved local linearity.
翻译:Adversarial培训(AT)及其变式是获得对抗性强型模型的最有效方法。AT的一个独特特点是,在更新模型重量之前,内部最大化问题需要反复解决,才能反复解决,才能更新模型重量,使培训速度缓慢。FGSMAT显著提高了效率,但当步数增长时却失败了。SOTA GradAlign使FGSMAT与一个更高的职级尺寸相匹配,然而,它在输入梯度上的正规化使其比FGSMAT慢3到4倍。我们提议的NoiseAug通过直接对输入本身进行常规化来消除额外的计算间接费用。这项工作的主要贡献在于经验性发现单步FGSM AT并非像以往工作路线所建议的那样艰难:噪音增强是你们快速前进所需要的。为了了解我们的NoiseAug的成功,我们进行了广泛的分析,发现减轻收缩过度(CO)和Robust Arabit (RO)需要不同的增加量。而不是由于数据增强而导致更多的样品,我们确定使NoiseAug公司在防止其本地增加的谎言方面变得有效。