Neural networks often make predictions relying on the spurious correlations from the datasets rather than the intrinsic properties of the task of interest, facing sharp degradation on out-of-distribution (OOD) test data. Existing de-bias learning frameworks try to capture specific dataset bias by annotations but they fail to handle complicated OOD scenarios. Others implicitly identify the dataset bias by special design low capability biased models or losses, but they degrade when the training and testing data are from the same distribution. In this paper, we propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model. The base model is encouraged to focus on examples that are hard to solve with biased models, thus remaining robust against spurious correlations in the test stage. GGD largely improves models' OOD generalization ability on various tasks, but sometimes over-estimates the bias level and degrades on the in-distribution test. We further re-analyze the ensemble process of GGD and introduce the Curriculum Regularization inspired by curriculum learning, which achieves a good trade-off between in-distribution and out-of-distribution performance. Extensive experiments on image classification, adversarial question answering, and visual question answering demonstrate the effectiveness of our method. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
翻译:在本文中,我们提议了一个GGD的“GGD”学习框架(GGD),这个框架贪婪地培训了偏向模式和基础模型。 基础模型鼓励侧重于那些难以用偏向模式解决的范例,从而在分配和透视阶段保持强健,以抵制虚假的关联。 GGD 基本上改进了模型在各种任务方面的OOD一般化能力,但有时高估了偏向水平,并降低了分配测试标准。我们进一步重新分析了GGGD的多重偏向性模型,并引入了课程学习所启发的课程正规化,从而实现了在分配和透视过程中的自我交易。