Existing natural language understanding (NLU) models often rely on dataset biases rather than intended task-relevant features to achieve high performance on specific datasets. As a result, these models perform poorly on datasets outside the training distribution. Some recent studies address the above issue by reducing the weights of biased samples during the training process. However, these methods still encode biased latent features in representations and neglect the dynamic nature of bias, which hinders model prediction. We propose an NLU debiasing method, named debiasing contrastive learning (DCT), to simultaneously alleviate the above problems based on contrastive learning. We devise a debiasing positive sampling strategy to mitigate biased latent features by selecting the least similar biased positive samples. We also propose a dynamic negative sampling strategy to capture the dynamic influence of biases by employing a bias-only model to dynamically select the most similar biased negative samples. We conduct experiments on three NLU benchmark datasets. Experimental results show that DCT outperforms state-of-the-art baselines on out-of-distribution datasets while maintaining in-distribution performance. We also verify that DCT can reduce biased latent features from the model's representations.
翻译:现有自然语言理解模式往往依赖于数据集偏差,而不是预期的任务相关特征,以在具体数据集上取得高绩效。因此,这些模型在培训分布之外,在数据集上表现不佳。最近的一些研究通过在培训过程中减少偏差样本的重量,解决了上述问题。然而,这些方法仍然将偏差潜在特征在表现上的偏差编码成,忽视偏见的动态性质,从而妨碍模型预测。我们提议了一种NLU偏差偏差方法,称为偏差对比学习(DCT),以根据对比性学习来同时缓解上述问题。我们制定了积极的抽样战略,通过选择最相似的偏差积极样本来减少偏差潜在特征。我们还提出了动态负面抽样战略,通过使用偏差模式来动态选择最相似的偏差负面样本来捕捉偏差的动态影响。我们在三个NLU基准数据集上进行实验。实验结果表明,DCT在保持分布模型的性性能的同时,DCT优于分配外数据集的最新基线。我们还核实DCT的偏差性表现。