Neural machine translation (NMT) is often criticized for failures that happen without awareness. The lack of competency awareness makes NMT untrustworthy. This is in sharp contrast to human translators who give feedback or conduct further investigations whenever they are in doubt about predictions. To fill this gap, we propose a novel competency-aware NMT by extending conventional NMT with a self-estimator, offering abilities to translate a source sentence and estimate its competency. The self-estimator encodes the information of the decoding procedure and then examines whether it can reconstruct the original semantics of the source sentence. Experimental results on four translation tasks demonstrate that the proposed method not only carries out translation tasks intact but also delivers outstanding performance on quality estimation. Without depending on any reference or annotated data typically required by state-of-the-art metric and quality estimation methods, our model yields an even higher correlation with human quality judgments than a variety of aforementioned methods, such as BLEURT, COMET, and BERTScore. Quantitative and qualitative analyses show better robustness of competency awareness in our model.
翻译:缺乏能力意识使得NMT不可信。这与在对预测有疑问时提供反馈或进行进一步调查的翻译者形成鲜明对比。为了填补这一空白,我们提议采用新的能力意识NMT, 扩大传统的NMT, 使用自测器, 提供翻译源句和估计其能力的能力。自我测量者对解码程序的信息进行编码,然后审查它是否能够重建源句的原始语义。四项翻译任务的实验结果表明,拟议的方法不仅未雨绸缪地完成翻译工作,而且还在质量估计方面提供出色的业绩。我们的模式不依赖任何参考或通常由最新指标和质量估计方法要求的附加数据,与上述各种方法相比,例如BLEURT、COMET和BERTScore。定量和定性分析表明,我们模型的能力意识更加牢固。