Prompting method is regarded as one of the crucial progress for few-shot nature language processing. Recent research on prompting moves from discrete tokens based ``hard prompts'' to continuous ``soft prompts'', which employ learnable vectors as pseudo prompt tokens and achieve better performance. Though showing promising prospects, these soft-prompting methods are observed to rely heavily on good initialization to take effect. Unfortunately, obtaining a perfect initialization for soft prompts requires understanding of inner language models working and elaborate design, which is no easy task and has to restart from scratch for each new task. To remedy this, we propose a generalized soft prompting method called MetaPrompting, which adopts the well-recognized model-agnostic meta-learning algorithm to automatically find better prompt initialization that facilitates fast adaptation to new prompting tasks.Extensive experiments show MetaPrompting tackles soft prompt initialization problem and brings significant improvement on four different datasets (over 6 points improvement in accuracy for 1-shot setting), achieving new state-of-the-art performance.
翻译:催动方法被认为是少数自然语言处理的关键进展之一。 最近关于促动从基于“硬提示”的离散象征到持续“软提示”的研究,该方法使用可学习的矢量作为假提示符号,并取得更好的性能。虽然前景光明,但观察到这些软提示方法在很大程度上依赖良好的初始化来发挥作用。不幸的是,为软提示获得完美的初始化需要理解内部语言模型的工作和详细设计,这并非易事,而且每件新任务都必须从零开始。为了纠正这一点,我们提议一种普遍软提示方法,称为“MetaPrompting”,采用公认的模型――不可知化的元学习算法,以自动找到更迅速的初始化方法,便于快速适应新的快速任务。广泛的实验显示,MetaPrompting解决软提示初始化问题,并大大改进四个不同的数据集(在一线设定时,精确度提高6个百分点以上),实现新的状态。