Recent studies show that deep neural networks (DNN) are vulnerable to adversarial examples, which aim to mislead DNNs by adding perturbations with small magnitude. To defend against such attacks, both empirical and theoretical defense approaches have been extensively studied for a single ML model. In this work, we aim to analyze and provide the certified robustness for ensemble ML models, together with the sufficient and necessary conditions of robustness for different ensemble protocols. Although ensemble models are shown more robust than a single model empirically; surprisingly, we find that in terms of the certified robustness the standard ensemble models only achieve marginal improvement compared to a single model. Thus, to explore the conditions that guarantee to provide certifiably robust ensemble ML models, we first prove that diversified gradient and large confidence margin are sufficient and necessary conditions for certifiably robust ensemble models under the model-smoothness assumption. We then provide the bounded model-smoothness analysis based on the proposed Ensemble-before-Smoothing strategy. We also prove that an ensemble model can always achieve higher certified robustness than a single base model under mild conditions. Inspired by the theoretical findings, we propose the lightweight Diversity Regularized Training (DRT) to train certifiably robust ensemble ML models. Extensive experiments show that our DRT enhanced ensembles can consistently achieve higher certified robustness than existing single and ensemble ML models, demonstrating the state-of-the-art certified L2-robustness on MNIST, CIFAR-10, and ImageNet datasets.
翻译:最近的研究显示,深层神经网络(DNN)很容易受到对抗性的例子的影响,这些例子的目的是通过增加小范围的扰动来误导DNN;为了防范这种攻击,已经为单一的ML模型广泛研究了实验性和理论防御方法。在这项工作中,我们的目标是分析和为混合的ML模型提供经认证的稳健性,同时为不同的混合协议提供足够和必要的稳健性条件。尽管混合模型比单一的经验模型更强;令人惊讶的是,我们发现标准混合模型在经认证的稳健性方面,与单一模型相比,只能实现边际改进。因此,为了探索保证提供可靠稳健的混合ML模型的条件,我们首先要证明,多样化的梯度和巨大的信任幅度是足够和必要的条件,在模型测算中,我们随后提供了基于拟议的超强的模型的模型;我们通过经认证的稳健性模型,我们还可以通过经认证的标准化的标准化的单一模型来显示一个稳定的标准化模型。