Recent progress in empirical and certified robustness promises to deliver reliable and deployable Deep Neural Networks (DNNs). Despite that success, most existing evaluations of DNN robustness have been done on images sampled from the same distribution that the model was trained on. Yet, in the real world, DNNs may be deployed in dynamic environments that exhibit significant distribution shifts. In this work, we take a first step towards thoroughly investigating the interplay between empirical and certified adversarial robustness on one hand and domain generalization on another. To do so, we train robust models on multiple domains and evaluate their accuracy and robustness on an unseen domain. We observe that: (1) both empirical and certified robustness generalize to unseen domains, and (2) the level of generalizability does not correlate well with input visual similarity, measured by the FID between source and target domains. We also extend our study to cover a real-world medical application, in which adversarial augmentation enhances both the robustness and generalization accuracy in unseen domains.
翻译:尽管取得了这一成功,但对DNN的稳健性的大多数现有评价都是对模型所培训的相同分布的图像进行的。然而,在现实世界中,DNN可能会在动态环境中部署,而动态环境中的分布变化显著。在这项工作中,我们迈出了第一步,彻底调查经验型和经认证的对立强性之间的相互作用和对域的概括性。为了做到这一点,我们在多个领域培养了强健模型,并评估了这些模型在无形领域的准确性和稳健性。我们注意到:(1) 经验型和经认证的稳健性普遍适用于无形领域,以及(2) 通用性水平与输入的视觉相似性没有很好的联系,而输入的视觉相似性是由FID在源和目标领域之间测量的。我们还将我们的研究扩大到涵盖真实世界的医疗应用,在这种应用中,对抗性增强增强对隐蔽域的稳健性和一般准确性。