Non-autoregressive neural machine translation (NAT) models suffer from the multi-modality problem that there may exist multiple possible translations of a source sentence, so the reference sentence may be inappropriate for the training when the NAT output is closer to other translations. In response to this problem, we introduce a rephraser to provide a better training target for NAT by rephrasing the reference sentence according to the NAT output. As we train NAT based on the rephraser output rather than the reference sentence, the rephraser output should fit well with the NAT output and not deviate too far from the reference, which can be quantified as reward functions and optimized by reinforcement learning. Experiments on major WMT benchmarks and NAT baselines show that our approach consistently improves the translation quality of NAT. Specifically, our best variant achieves comparable performance to the autoregressive Transformer, while being 14.7 times more efficient in inference.
翻译:非偏向性神经机车翻译模型存在多时制问题,即源句可能存在多种可能的翻译,因此当NAT输出接近其他翻译时,参考句可能不适合培训。针对这一问题,我们引入了重新措辞器,通过根据NAT输出改写参考句,为NAT提供更好的培训目标。随着我们根据改写输出而不是参考句进行NAT培训,改写器输出应与NAT输出相适应,不要偏离参考段,因为参考段可以量化为奖励功能,并通过强化学习优化。关于主要WMT基准和NAT基线的实验表明,我们的方法一贯提高NAT的翻译质量。具体地说,我们的最佳变式实现了与自动反向变换换的类似性能,而引证效率是14.7倍。