Providing pretrained language models with simple task descriptions or prompts in natural language yields impressive few-shot results for a wide range of text classification tasks when combined with gradient-based learning from examples. In this paper, we show that the underlying idea can also be applied to text generation tasks: We adapt Pattern-Exploiting Training (PET), a recently proposed few-shot approach, for finetuning generative language models on text generation tasks. On several text summarization and headline generation datasets, our proposed variant of PET gives consistent improvements over a strong baseline in few-shot settings.
翻译:提供具有简单任务描述或自然语言提示的预先培训语言模型,在结合基于梯度的从实例中学习的同时,为一系列广泛的文本分类任务带来令人印象深刻的微小结果。在本文中,我们表明这一基本想法也可以适用于文本生成任务:我们改编了“模式开发培训”,这是最近提出的一个微小方法,用于微调关于文本生成任务的变异语言模型。关于几个文本归纳和标题生成数据集,我们拟议的“模式开发”变式在几个发件环境的强势基线上给出了一致的改进。