Despite significant progress in the quality of language generated from abstractive summarization models, these models still exhibit the tendency to hallucinate, i.e., output content not supported by the source document. A number of works have tried to fix--or at least uncover the source of--the problem with limited success. In this paper, we identify a simple criterion under which models are significantly more likely to assign more probability to hallucinated content during generation: high model uncertainty. This finding offers a potential explanation for hallucinations: models default to favoring text with high marginal probability, i.e., high-frequency occurrences in the training set, when uncertain about a continuation. It also motivates possible routes for real-time intervention during decoding to prevent such hallucinations. We propose a decoding strategy that switches to optimizing for pointwise mutual information of the source and target token--rather than purely the probability of the target token--when the model exhibits uncertainty. Experiments on the XSum dataset show that our method decreases the probability of hallucinated tokens while maintaining the Rouge and BertS scores of top-performing decoding strategies.
翻译:尽管抽象概括模型在语言质量方面取得了显著进步,但这些模型仍然呈现出幻觉的倾向,即产出内容不受源文件支持。一些作品试图修复或至少发现问题的根源,但成效有限。在本文中,我们确定了一个简单标准,根据这一标准,模型在生成过程中极有可能给幻觉内容分配更多概率:高模型不确定性。这一发现为幻觉提供了潜在解释:模型默认偏好文本的概率极小,即当对延续性不确定时,培训集中出现高频率现象。它还在解码防止此类幻觉的过程中鼓励可能的实时干预途径。我们提议了一个解码战略,以优化源和目标符号的原始相互信息,而不是模型显示不确定性时目标符号的纯概率。XSum数据集的实验表明,我们的方法在保持红色和BertS最高表现解密战略的分数的同时,降低了粉末标记的概率。