Translation Quality Estimation (QE) is the task of predicting the quality of machine translation (MT) output without any reference. This task has gained increasing attention as an important component in practical applications of MT. In this paper, we first propose XLMRScore, a simple unsupervised QE method based on the BERTScore computed using the XLM-RoBERTa (XLMR) model while discussing the issues that occur using this method. Next, we suggest two approaches to mitigate the issues: replacing untranslated words with the unknown token and the cross-lingual alignment of pre-trained model to represent aligned words closer to each other. We evaluate the proposed method on four low-resource language pairs of WMT21 QE shared task, as well as a new English-Farsi test dataset introduced in this paper. Experiments show that our method could get comparable results with the supervised baseline for two zero-shot scenarios, i.e., with less than 0.01 difference in Pearson correlation, while outperforming the unsupervised rivals in all the low-resource language pairs for above 8% in average.
翻译:翻译质量估计( QE) 是预测机器翻译( MT) 输出质量而无需参考的任务 。 这项任务作为MT 实际应用中的一个重要部分, 日益引起人们的关注 。 在本文中, 我们首先提出 XLMRScore, 这是一种简单且不受监督的 QE 方法, 其依据是使用 XLM- ROBERTA (XLMR) 模型计算 的 BERTScore 计算 的 简单 QE 方法, 并同时讨论 使用此方法发生的问题 。 其次, 我们建议了两种办法来缓解问题: 替换未翻译的单词, 代之以未知的符号, 以及 将预培训模式的跨语种对齐, 以代表彼此相近的单词 。 我们评估了 WMT21 QE 共享的四对低资源语言的拟议方法, 以及本文中引入的一个新的英法西测试数据集 。 实验显示, 我们的方法可以与两种零度假设的基线相比, 不到0.01 差异,,, 皮尔森, 的对比差差差,, 超过 8 % 。