The performance of state-of-the-art neural rankers can deteriorate substantially when exposed to noisy inputs or applied to a new domain. In this paper, we present a novel method for fine-tuning neural rankers that can significantly improve their robustness to out-of-domain data and query perturbations. Specifically, a contrastive loss that compares data points in the representation space is combined with the standard ranking loss during fine-tuning. We use relevance labels to denote similar/dissimilar pairs, which allows the model to learn the underlying matching semantics across different query-document pairs and leads to improved robustness. In experiments with four passage ranking datasets, the proposed contrastive fine-tuning method obtains improvements on robustness to query reformulations, noise perturbations, and zero-shot transfer for both BERT and BART based rankers. Additionally, our experiments show that contrastive fine-tuning outperforms data augmentation for robustifying neural rankers.
翻译:最先进的神经排层器的性能在受到噪音输入或应用到一个新领域时可能会大幅恶化。 在本文中,我们提出了一个微调神经排层器的新型方法,可以大大提高其对外部数据和查询扰动的稳健性。 具体地说,将代表空间中的数据点进行比较的对比性损失与微调过程中的标准排位损失结合起来。 我们使用相关标签来表示相近/不同的配对,使模型能够学习不同查询文件对对等的基本对应语义,并导致增强稳健性。 在四个段落排名数据集的实验中,提议的对比性微调方法在对重整、噪音扰动和BERT和BART的排层进行零光传输的稳健性方面得到了改进。 此外,我们的实验显示,对比性微调外形数据增强强力神经排层的增强值。