Neural machine translation (NMT) has become the de-facto standard in real-world machine translation applications. However, NMT models can unpredictably produce severely pathological translations, known as hallucinations, that seriously undermine user trust. It becomes thus crucial to implement effective preventive strategies to guarantee their proper functioning. In this paper, we address the problem of hallucination detection in NMT by following a simple intuition: as hallucinations are detached from the source content, they exhibit encoder-decoder attention patterns that are statistically different from those of good quality translations. We frame this problem with an optimal transport formulation and propose a fully unsupervised, plug-in detector that can be used with any attention-based NMT model. Experimental results show that our detector not only outperforms all previous model-based detectors, but is also competitive with detectors that employ large models trained on millions of samples.
翻译:神经机器翻译(NMT)已经成为现实世界机器翻译应用程序的脱法标准。 但是,NMT模型无法预测地能够产生严重病理翻译,称为幻觉,严重损害用户的信任。因此,执行有效的预防战略以保障其正常运转变得至关重要。 在本文中,我们通过简单的直觉来解决NMT中幻觉检测的问题:由于幻觉与源内容脱钩,它们表现出在统计上不同于高质量翻译的编码脱解关注模式。我们用一种最佳的运输配方来解决这个问题,并提出一种完全不受监督的插件检测器,可以与任何以关注为基础的NMT模型一起使用。实验结果显示,我们的探测器不仅超越了以往所有基于模型的探测器,而且与使用以数百万样本为主的大型模型的探测器竞争。