Multiple intriguing problems are hovering in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off. These problems pose great challenges to both reliable evaluation and practical deployment. Here, we empirically show that these problems share one common cause -- low-quality samples in the dataset. Specifically, we first propose a strategy to measure the data quality based on the learning behaviors of the data during adversarial training and find that low-quality data may not be useful and even detrimental to the adversarial robustness. We then design controlled experiments to investigate the interconnections between data quality and problems in adversarial training. We find that when low-quality data is removed, robust overfitting and robustness overestimation can be largely alleviated; and robustness-accuracy trade-off becomes less significant. These observations not only verify our intuition about data quality but may also open new opportunities to advance adversarial training.
翻译:在对抗性培训中,许多令人感兴趣的问题都徘徊在对抗性培训中,包括强力的超配、强力的过高估计和稳健的准确性权衡。这些问题对可靠的评价和实际部署都构成巨大的挑战。在这里,我们从经验上表明,这些问题有一个共同的原因 -- -- 数据集中的低质量样本。具体地说,我们首先提出一项战略,根据对抗性培训期间的数据学习行为衡量数据质量,发现低质量数据可能没有用处,甚至对对抗性强健性有害。然后我们设计有控制的实验,以调查数据质量与对抗性培训中的问题之间的相互联系。我们发现,当低质量数据被删除、稳健的超配和稳健的过高估计可以在很大程度上得到缓解;稳健的准确性交易变得不那么重要。这些观察不仅证实了我们对数据质量的直觉,而且还可能为推进对抗性培训开辟了新的机会。