In this work we study the robustness to adversarial attacks, of early-stopping strategies on gradient-descent (GD) methods for linear regression. More precisely, we show that early-stopped GD is optimally robust (up to an absolute constant) against Euclidean-norm adversarial attacks. However, we show that this strategy can be arbitrarily sub-optimal in the case of general Mahalanobis attacks. This observation is compatible with recent findings in the case of classification~\cite{Vardi2022GradientMP} that show that GD provably converges to non-robust models. To alleviate this issue, we propose to apply instead a GD scheme on a transformation of the data adapted to the attack. This data transformation amounts to apply feature-depending learning rates and we show that this modified GD is able to handle any Mahalanobis attack, as well as more general attacks under some conditions. Unfortunately, choosing such adapted transformations can be hard for general attacks. To the rescue, we design a simple and tractable estimator whose adversarial risk is optimal up to within a multiplicative constant of 1.1124 in the population regime, and works for any norm.
翻译:在这项工作中,我们研究了关于梯度-白(GD)线性回归方法的早期制止战略对对抗性攻击的稳健性。更准确地说,我们表明,早期制止GD对于Euclidean-Norm对抗性攻击是最佳强健(直至绝对恒定)的。然而,我们表明,在马哈拉诺比斯将军的攻击中,这一战略可以任意地低于最佳程度。这一观察与最近关于“cite {Vardi2022GradientMP”的分类中发现的结果相容,这表明GD可明显地与非野蛮模式汇合。为了缓解这一问题,我们提议采用一个GD计划来改造适应攻击性攻击性攻击的数据。这一数据转换相当于应用功能性调整学习率,我们表明,在马哈拉诺比斯将军的攻击中,以及在某些条件下,这个经过修改的GD能够任意地处理任何马哈拉诺比攻击。不幸的是,选择这种经过调整的变换的变换方法对于一般攻击来说可能是困难的。对于救援来说,我们设计了一个简单和可移动的估测的估测算器,其对抗风险在多一一一至一一至一一一一一一一一九一九的系统内的最佳。