Commonsense generation is a challenging task of generating a plausible sentence describing an everyday scenario using provided concepts. Its requirement of reasoning over commonsense knowledge and compositional generalization ability even puzzles strong pre-trained language generation models. We propose a novel framework using retrieval methods to enhance both the pre-training and fine-tuning for commonsense generation. We retrieve prototype sentence candidates by concept matching and use them as auxiliary input. For fine-tuning, we further boost its performance with a trainable sentence retriever. We demonstrate experimentally on the large-scale CommonGen benchmark that our approach achieves new state-of-the-art results.
翻译:常识一代是一项具有挑战性的任务,即利用提供的概念来生成一个可信的句子,描述日常生活情景。它要求对常识知识进行推理,并采用通用的构成能力,这甚至使经过训练的强有力的语言生成模型产生疑惑。我们提出了一个新的框架,利用检索方法加强常识一代的培训前和微调。我们通过概念匹配来检索原型判决候选人,并将他们用作辅助投入。为了微调,我们用一个可训练的句子检索器来进一步提升其表现。我们实验性地展示了大规模常识基准,即我们的方法取得了新的最新成果。