通过最优化交通进行词汇学习,用于机器翻译 (Vocabulary Learning via Optimal Transport for Machine Translation)

The choice of token vocabulary affects the performance of machine translation. This paper aims to figure out what is a good vocabulary and whether one can find the optimal vocabulary without trial training. To answer these questions, we first provide an alternative understanding of the role of vocabulary from the perspective of information theory. Motivated by this, we formulate the quest of vocabularization -- finding the best token dictionary with a proper size -- as an optimal transport (OT) problem. We propose VOLT, a simple and efficient solution without trial training. Empirical results show that VOLT outperforms widely-used vocabularies in diverse scenarios, including WMT-14 English-German and TED's 52 translation directions. For example, VOLT achieves almost 70% vocabulary size reduction and 0.5 BLEU gain on English-German translation. Also, compared to BPE-search, VOLT reduces the search time from 384 GPU hours to 30 GPU hours on English-German translation. Codes are available at https://github.com/Jingjing-NLP/VOLT .

翻译：选择象征性词汇会影响机器翻译的性能。本文旨在找出什么是好的词汇, 以及人们能否在没有试用训练的情况下找到最佳词汇。为了解答这些问题, 我们首先从信息理论的角度对词汇的作用提供另一种理解。我们受此驱动, 将寻找Vocabulalization -- -- 找到具有适当尺寸的最佳象征性字典 -- -- 作为一种最佳运输( OT) 问题。我们建议VOLT, 一种不经过试用培训的简单而有效的解决方案。经验性结果显示, VOLT 超越了不同情景中广泛使用的词汇, 包括 WMT-14 英文- 德文和TED 52 的翻译方向。例如, VOLT 实现了几乎70%的词汇缩放规模和英语- 德文翻译的0.5 BLEU 增益。此外, 与 BPE- 搜索相比, VOLT 将搜索时间从384 GPU小时减少到英语- 德语翻译的30 GPU小时。代码可在 https://github. com/ Jingjing- NLP/ VOLT 。