Notwithstanding the promise of Lipschitz-based approaches to \emph{deterministically} train and certify robust deep networks, the state-of-the-art results only make successful use of feed-forward Convolutional Networks (ConvNets) on low-dimensional data, e.g. CIFAR-10. Because ConvNets often suffer from vanishing gradients when going deep, large-scale datasets with many classes, e.g., ImageNet, have remained out of practical reach. This paper investigates ways to scale up certifiably robust training to Residual Networks (ResNets). First, we introduce the \emph{Linear ResNet} (LiResNet) architecture, which utilizes a new residual block designed to facilitate \emph{tighter} Lipschitz bounds compared to a conventional residual block. Second, we introduce Efficient Margin MAximization (EMMA), a loss function that stabilizes robust training by simultaneously penalizing worst-case adversarial examples from \emph{all} classes. Combining LiResNet and EMMA, we achieve new \emph{state-of-the-art} robust accuracy on CIFAR-10/100 and Tiny-ImageNet under $\ell_2$-norm-bounded perturbations. Moreover, for the first time, we are able to scale up deterministic robustness guarantees to ImageNet, bringing hope to the possibility of applying deterministic certification to real-world applications.
翻译:尽管基于Lipschitz(Lipschitz)的培训和认证强大深度网络(ResNets)的做法很有希望,但最新的结果只是成功地利用低维数据(如CIFAR-10)的Feed-forward Convolutional Nets(ConventalNets) (CondNets) (Conf-forward Convolutional Nets) (Convilal Nets) (Convilal Nets) (Convilal Nets) (Convilational Nets) (Convilational Nets) (Convild) (Convild-form) (Confreat-Conformation) (Confront-Creative) (Conflation) (Conferrence) (Conferrlation) (Lipschitzs) ) (因为Confload the losemental-late, maciscial Maxm) (ement) (e,我们引入了高效的IM2-I.I-I) (eal-I-I.I.I.I.I) (I) (I) (I.) (I.I) (i) (I) (I) (I)) (I.)) (I-I-I-I-I-I) int-I.) (I.) 和Emtalviolfirm) (I.) (I) (I.) (I.) (I) (I.) (I-I-I.) (I-I-I-I-I-I-I-I-I-I-I-I-) ) ) (在)) (I-I) (I) (在) ) (I) (I) vicild) vicl) (I-I-I-I) vicl) (I-I-I-I-I-I-I-I-I-I) vi) vi) vi) vicl) (I) vicl) vi