Recent advances in large-scale pre-training such as GPT-3 allow seemingly high quality text to be generated from a given prompt. However, such generation systems often suffer from problems of hallucinated facts, and are not inherently designed to incorporate useful external information. Grounded generation models appear to offer remedies, but their training typically relies on rarely-available parallel data where information-relevant documents are provided for context. We propose a framework that alleviates this data constraint by jointly training a grounded generator and document retriever on the language model signal. The model learns to reward retrieval of the documents with the highest utility in generation, and attentively combines them using a Mixture-of-Experts (MoE) ensemble to generate follow-on text. We demonstrate that both generator and retriever can take advantage of this joint training and work synergistically to produce more informative and relevant text in both prose and dialogue generation.
翻译:在诸如GPT-3等大规模培训前阶段的最近进展中,从某一时刻就能产生出质量似乎很高的文本,然而,这类生成系统往往会遇到幻觉事实的问题,而且并非内在设计上能够纳入有用的外部信息。有源的生成模型似乎提供了补救措施,但其培训通常依赖于提供与背景有关文件的极少获得的平行数据。我们建议了一个框架,通过在语言模型信号上联合培训一个有根有据的生成器和文件检索器来缓解这一数据限制。模型学会奖励具有最高效用的一代文件的检索,并且利用混合探索者(MOE)的合用词仔细地将它们结合起来产生后续文本。我们证明,生成者和检索者都可以利用这一联合培训和协同工作,在源和对话生成中产生更多资料和相关的文本。