Retrieval-augmented generation models have shown state-of-the-art performance across many knowledge-intensive NLP tasks such as open question answering and fact verification. These models are trained to generate the final output given the retrieved passages, which can be irrelevant to the original query, leading to learning spurious cues or answer memorization. This work introduces a method to incorporate the evidentiality of passages -- whether a passage contains correct evidence to support the output -- into training the generator. We introduce a multi-task learning framework to jointly generate the final output and predict the evidentiality of each passage, leveraging a new task-agnostic method to obtain silver evidentiality labels for supervision. Our experiments on five datasets across three knowledge-intensive tasks show that our new evidentiality-guided generator significantly outperforms its direct counterpart with the same-size model and advances the state of the art on FaVIQ-Ambig. We attribute these improvements to both the auxiliary multi-task learning and silver evidentiality mining techniques.
翻译:重新获取强化的生成模型显示,许多知识密集型NLP任务(如开放式答题和事实核实)的先进性能,如开放式答题和事实核实。这些模型经过培训,根据检索到的段落生成最终产出,这与原始查询无关,导致学习虚假的提示或回答记忆。这项工作引入了一种方法,将通道的证据性 -- -- 某一段是否包含支持输出的正确证据 -- -- 纳入到生成器的培训中。我们引入了一个多任务学习框架,以联合生成最终输出并预测每一通道的表面性,利用一种新的任务识别方法获取银色证据标签以进行监督。我们在五个数据集上进行的实验显示,我们新的证据引导生成器与同一大小的模型大大超越了直接对应方,并推进了FaVIQ-Ambig的艺术状态。我们将这些改进归功于辅助性多任务学习和银色显性采矿技术。