We propose a metric -- Projection Norm -- to predict a model's performance on out-of-distribution (OOD) data without access to ground truth labels. Projection Norm first uses model predictions to pseudo-label test samples and then trains a new model on the pseudo-labels. The more the new model's parameters differ from an in-distribution model, the greater the predicted OOD error. Empirically, our approach outperforms existing methods on both image and text classification tasks and across different network architectures. Theoretically, we connect our approach to a bound on the test error for overparameterized linear models. Furthermore, we find that Projection Norm is the only approach that achieves non-trivial detection performance on adversarial examples. Our code is available at https://github.com/yaodongyu/ProjNorm.
翻译:我们提出一个衡量标准 -- -- 诺姆投影 -- -- 来预测模型在无法获取地面真象标签的情况下在分配外数据上的性能。诺姆投影首先将模型预测用于假标签测试样品,然后在假标签上培训新的模型。新模型的参数与分布内模型越不同,预测的OOOD误差就越大。我们的方法在图像和文本分类任务以及不同网络结构方面优于现有方法。理论上,我们将我们的方法与过度参数化线性模型测试错误的界限联系起来。此外,我们发现诺姆投影是唯一在对抗性例子中实现非三角探测性表现的方法。我们的代码可在https://github.com/yaodongyu/ProjNorm上查阅。