Word alignment is essential for the down-streaming cross-lingual language understanding and generation tasks. Recently, the performance of the neural word alignment models has exceeded that of statistical models. However, they heavily rely on sophisticated translation models. In this study, we propose a super lightweight unsupervised word alignment (SLUA) model, in which bidirectional symmetric attention trained with a contrastive learning objective is introduced, and an agreement loss is employed to bind the attention maps, such that the alignments follow mirror-like symmetry hypothesis. Experimental results on several public benchmarks demonstrate that our model achieves competitive, if not better, performance compared to the state of the art in word alignment while significantly reducing the training and decoding time on average. Further ablation analysis and case studies show the superiority of our proposed SLUA. Notably, we recognize our model as a pioneer attempt to unify bilingual word embedding and word alignments. Encouragingly, our approach achieves 16.4x speedup against GIZA++, and 50x parameter compression} compared with the Transformer-based alignment methods. We will release our code to facilitate the community.
翻译:单词对齐对于跨语言理解和生成任务至关重要。 最近,神经字对齐模型的性能超过了统计模型的性能。 但是,它们在很大程度上依赖复杂的翻译模型。 在本研究中,我们提议采用超轻轻轻轻的、不受监督的单词对齐模型(SLUA),在其中引入双向对称关注,以对比学习目标为培训,并采用协议损失来绑定关注地图,使对齐方法遵循镜像的对称假设。几个公共基准的实验结果显示,我们模型的性能与艺术的文字对齐状态相比,如果不是更好的话,也有竞争力,同时平均大幅减少培训和解码时间。进一步的缩略分析和案例研究显示,我们提议的SLUA的优越性。值得注意的是,我们承认我们的模型是试图统一双语词嵌入和单词对齐的先驱。令人鼓舞的是,我们的方法比变换后的校正方法达到了16.4x速度,比GIZA++和50x参数压缩}。我们将发布我们的代码,以促进社区。