Question answering models commonly have access to two sources of "knowledge" during inference time: (1) parametric knowledge - the factual knowledge encoded in the model weights, and (2) contextual knowledge - external knowledge (e.g., a Wikipedia passage) given to the model to generate a grounded answer. Having these two sources of knowledge entangled together is a core issue for generative QA models as it is unclear whether the answer stems from the given non-parametric knowledge or not. This unclarity has implications on issues of trust, interpretability and factuality. In this work, we propose a new paradigm in which QA models are trained to disentangle the two sources of knowledge. Using counterfactual data augmentation, we introduce a model that predicts two answers for a given question: one based on given contextual knowledge and one based on parametric knowledge. Our experiments on the Natural Questions dataset show that this approach improves the performance of QA models by making them more robust to knowledge conflicts between the two knowledge sources, while generating useful disentangled answers.
翻译:问题解答模型通常在推论时间可以接触到两种“知识”来源:(1) 参数知识—— 模型重量中编码的事实知识,(2) 背景知识—— 给模型的外部知识(例如维基百科段落),以产生一个有根有据的答案。这两个知识来源相互交织在一起,是基因化质量评估模型的一个核心问题,因为不清楚答案是否来自给定的非参数知识。这种模糊性对信任、可解释性和事实质量问题有影响。在这项工作中,我们提出了一个新的范例,对质量评估模型进行培训,以解开两个知识来源。我们利用反事实数据增强,我们引入了一种模型,预测一个问题有两个答案:一个基于特定背景知识,一个基于参数知识。我们对自然问题数据集的实验表明,这一方法通过提高质量评估模型在两个知识来源之间的知识冲突方面的强度,同时产生有用的解析性答案,从而改进了质量评估模型的性能。