Large pre-trained language models have brought remarkable progress in NLP. Pre-training and Fine-tuning have given state-of-art performance across tasks in text processing. Data Augmentation techniques have also helped build state-of-art models on low or zero resource tasks. Many works in the past have attempted at learning a single massively-multilingual machine translation model for zero-shot translation. Although those translation models are producing correct translations, the main challenge is those models are producing the wrong languages for zero-shot translation. This work and its results indicate that prompt conditioned large models do not suffer from off-target language errors i.e. errors arising due to translation to wrong languages. We empirically demonstrate the effectiveness of self-supervised pre-training and data augmentation for zero-shot multi-lingual machine translation.
翻译:预先培训的大型语言模型在NLP方面取得了显著进展。 预培训和微调使文本处理中的各项任务取得了最先进的业绩。数据增强技术还帮助建立了低资源或零资源任务的最新模型。过去许多工作曾尝试为零点翻译学习一个单一的大规模多语种机器翻译模型。虽然这些翻译模型正在产生正确的翻译,但主要的挑战在于这些模型为零点点翻译制作错误的语言。这项工作及其结果显示,即即时附加条件的大模型不会因非目标语言错误(即翻译错误语言引起的错误)而受到影响。我们从经验上证明,自我监督的培训和数据增强对于零点点数多语机器翻译是有效的。