The ability to learn from limited data, or few-shot learning, is a desirable and often critical requirement for NLP systems. While many existing methods do poorly at learning from a handful of examples, large pretrained language models have recently been shown to be efficient few-shot learners. One approach to few-shot learning, which does not require finetuning of model parameters, is to augment the language model's input with priming text which is typically constructed using task specific descriptions and examples. In this work, we further explore priming-based few-shot learning, with focus on using examples as prompts. We show that presenting examples in the right order is key for generalization. We introduce PERO (Prompting with Examples in the Right Order), where we formulate few-shot learning as search over the set of permutations of the training examples. We show that PERO can learn to generalize efficiently using as few as 10 examples, in contrast to existing approaches. While the newline token is a natural choice for separating the examples in the prompt, we show that learning a new separator token can potentially provide further gains in performance. We demonstrate the effectiveness of the proposed method on the tasks of sentiment classification, natural language inference and fact retrieval. Finally, we analyze the learned prompts to reveal novel insights, including the idea that two training examples in the right order alone can provide competitive performance for sentiment classification and natural language inference.
翻译:从有限的数据中学习的能力,或少见的学习,对于国家学习计划系统来说,是可取的,而且往往是关键的要求。虽然许多现有方法在从少数例子中学习方面做得不好,但许多经过预先训练的语言模式最近被证明是效率微小的学习者。一个不需要微调模型参数的微小学习方法,是用通常使用具体任务描述和实例构建的边缘文本来补充语言模式的投入。在这项工作中,我们进一步探索以微小的点为主的小点学习,重点是以实例作为提示。我们表明,在正确顺序中展示实例是普遍化的关键。我们引入了PERO(采用正确秩序中的例子),我们用微小的学习作为搜索成套培训范例的组合。我们表明,与现有方法相比,PERO可以学习以10个例子来有效地概括语言模式。虽然新线的标志是快速区分实例的一种自然选择,但我们表明,学习一个新的分隔符物标记可以进一步提高绩效。我们用正确的语言来分析自然意识,我们最后在正确的语言分类中可以分析自然意识,我们用正确的理解了对正确语言的判断方法的正确表现。