Despite the progress in machine translation quality estimation and evaluation in the last years, decoding in neural machine translation (NMT) is mostly oblivious to this and centers around finding the most probable translation according to the model (MAP decoding), approximated with beam search. In this paper, we bring together these two lines of research and propose quality-aware decoding for NMT, by leveraging recent breakthroughs in reference-free and reference-based MT evaluation through various inference methods like $N$-best reranking and minimum Bayes risk decoding. We perform an extensive comparison of various possible candidate generation and ranking methods across four datasets and two model classes and find that quality-aware decoding consistently outperforms MAP-based decoding according both to state-of-the-art automatic metrics (COMET and BLEURT) and to human assessments. Our code is available at https://github.com/deep-spin/qaware-decode.
翻译:尽管过去几年在机器翻译质量估计和评价方面取得了进展,但神经机翻译的解码工作大多忽略了这一点,并围绕根据模型找到最有可能的翻译(MAP解码),与光束搜索相近。在本文中,我们汇集了这两条研究线,并提出了NMT质量认知解码建议,利用最近通过各种推理方法在无参考和基于参考的MT评估方面取得的突破,例如用美元最佳重排和最低贝斯风险解码等。我们广泛比较了四个数据集和两个模型类中各种可能的候选生成和排序方法,发现质量自觉解码始终超越MAP的解码,按照最先进的自动指标(COMET和BLEURT)和人类评估进行。我们的代码可在https://github.com/deep-spin/qawe-decode。我们可在https://githuu.com/deep-spin/qawe-deco中查阅。