The vulnerability of machine learning models to spurious correlations has mostly been discussed in the context of supervised learning (SL). However, there is a lack of insight on how spurious correlations affect the performance of popular self-supervised learning (SSL) and auto-encoder based models (AE). In this work, we shed light on this by evaluating the performance of these models on both real world and synthetic distribution shift datasets. Following observations that the linear head itself can be susceptible to spurious correlations, we develop a novel evaluation scheme with the linear head trained on out-of-distribution (OOD) data, to isolate the performance of the pre-trained models from a potential bias of the linear head used for evaluation. With this new methodology, we show that SSL models are consistently more robust to distribution shifts and thus better at OOD generalisation than AE and SL models.
翻译:机器学习模型在虚假关联方面的脆弱性,大部分是在监督学习的背景下讨论的(SL),然而,对于虚假关联如何影响以自我监督学习和自动编码为基础的模型(AE)的绩效,我们缺乏洞察力。在这项工作中,我们通过在真实世界和合成分布转换数据集中评价这些模型的绩效来阐明这一点。在观察到线性头本身容易受到虚假关联的情况下,我们与受过关于传播外数据培训的线性头(OOOD)开发了一个新的评价计划,以将预先培训的模型的性能与用于评价的线性头的潜在偏差区分开来。我们用这一新的方法表明,SLSL模型一贯地更强有力地适应分布变化,从而在OD通用方面比AE和SL模型更好。