Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. But what exactly in the training data causes a model to make a certain prediction? We seek to answer this question by providing a language for describing how training data influences predictions, through a causal framework. Importantly, our framework bypasses the need to retrain expensive models and allows us to estimate causal effects based on observational data alone. Addressing the problem of extracting factual knowledge from pretrained language models (PLMs), we focus on simple data statistics such as co-occurrence counts and show that these statistics do influence the predictions of PLMs, suggesting that such models rely on shallow heuristics. Our causal framework and our results demonstrate the importance of studying datasets and the benefits of causality for understanding NLP models.
翻译:大量培训数据是高水平NLP模型高性能的主要原因之一。但是,培训数据中的确切原因是什么促使一种模型进行某种预测?我们试图通过提供一种语言来说明培训数据如何通过因果框架影响预测来回答这个问题。重要的是,我们的框架绕过了重新培训昂贵模型的需要,使我们可以仅根据观察数据来估计因果关系。我们处理从预先培训的语言模型中提取事实知识的问题,我们注重简单的数据统计,例如共同发生统计,并表明这些统计确实影响PLM的预测,表明这些模型依赖浅层的理论。我们的因果框架和我们的结果表明研究数据集的重要性以及理解NLP模型的因果关系的好处。