Nearest neighbor machine translation augments the Autoregressive Translation~(AT) with $k$-nearest-neighbor retrieval, by comparing the similarity between the token-level context representations of the target tokens in the query and the datastore. However, the token-level representation may introduce noise when translating ambiguous words, or fail to provide accurate retrieval results when the representation generated by the model contains indistinguishable context information, e.g., Non-Autoregressive Translation~(NAT) models. In this paper, we propose a novel $n$-gram nearest neighbor retrieval method that is model agnostic and applicable to both AT and NAT models. Specifically, we concatenate the adjacent $n$-gram hidden representations as the key, while the tuple of corresponding target tokens is the value. In inference, we propose tailored decoding algorithms for AT and NAT models respectively. We demonstrate that the proposed method consistently outperforms the token-level method on both AT and NAT models as well on general as on domain adaptation translation tasks. On domain adaptation, the proposed method brings $1.03$ and $2.76$ improvements regarding the average BLEU score on AT and NAT models respectively.
翻译:近距离近距离机器翻译增加了自动递增翻译~(AT), 增加了美元- 美元- 美元- 美元- 美元- 美元- 美元- 近邻检索, 比较了查询和数据储存中目标符号的象征性背景表示和数据储存中的目标符号之间的相似性。 但是, 象征性表示在翻译模棱两可的单词时可能会出现噪音, 或者当模型生成的表示包含无法区分的上下文信息时无法提供准确的检索结果, 例如, 非自动递增翻译~ (NAT) 模型。 在本文中, 我们提议了一个小说 $- 美元- 克 最近的近邻检索方法, 其模型为AT 和 NAT 模型的模拟, 并且适用于AT 和 NAT 模型的普通化, 分别将相邻的 $ 美元- 隐藏表示为关键值, 而相应的目标符号的图示则是价值。 我们推断, 我们建议为AT AT 和 NAT 平方 平方 和 平方 平方 平方平方平方平方平方平方平方平方平方平方平方平方平方平方平方平方平方平方平方平方平调。