Large language models (LLMs) that have been trained on multilingual but not parallel text exhibit a remarkable ability to translate between languages. We probe this ability in an in-depth study of the pathways language model (PaLM), which has demonstrated the strongest machine translation (MT) performance among similarly-trained LLMs to date. We investigate various strategies for choosing translation examples for few-shot prompting, concluding that example quality is the most important factor. Using optimized prompts, we revisit previous assessments of PaLM's MT capabilities with more recent test sets, modern MT metrics, and human evaluation, and find that its performance, while impressive, still lags that of state-of-the-art supervised systems. We conclude by providing an analysis of PaLM's MT output which reveals some interesting properties and prospects for future work.
翻译:在多语种但非平行文本方面受过培训的大型语言模型(LLMs)表现出不同语言之间翻译的非凡能力。我们在对路径语言模型(PALM)的深入研究中发现了这一能力,该模型迄今为止在受过类似培训的LLMs中表现出最强的机器翻译(MT)性能。我们调查了为几发短片选择翻译示例的各种战略,得出了这一示例质量是最重要的因素。我们利用优化的提示,重新审视了以前对PALM的MT能力的评估,采用了最新的测试、现代MT指标和人文评估,发现其绩效虽然令人印象深刻,但仍然落后于最先进的监督系统。我们最后通过对PALM的MM输出提供了分析,揭示了未来工作的一些有趣的特性和前景。