Language Models (LMs) have proven to be useful in various downstream applications, such as summarisation, translation, question answering and text classification. LMs are becoming increasingly important tools in Artificial Intelligence, because of the vast quantity of information they can store. In this work, we present ProP (Prompting as Probing), which utilizes GPT-3, a large Language Model originally proposed by OpenAI in 2020, to perform the task of Knowledge Base Construction (KBC). ProP implements a multi-step approach that combines a variety of prompting techniques to achieve this. Our results show that manual prompt curation is essential, that the LM must be encouraged to give answer sets of variable lengths, in particular including empty answer sets, that true/false questions are a useful device to increase precision on suggestions generated by the LM, that the size of the LM is a crucial factor, and that a dictionary of entity aliases improves the LM score. Our evaluation study indicates that these proposed techniques can substantially enhance the quality of the final predictions: ProP won track 2 of the LM-KBC competition, outperforming the baseline by 36.4 percentage points. Our implementation is available on https://github.com/HEmile/iswc-challenge.
翻译:事实证明,语言模型(LMS)在诸如总结、翻译、问答和文本分类等各种下游应用中非常有用。LMS由于能够储存大量信息,在人工智能中越来越成为重要工具。在这项工作中,我们介绍了ProP(Promping as Probing),它使用GPT-3,这是OpenAI于2020年提出的一个大型语言模型,用于履行知识基础建设的任务。ProP采取多步方法,结合各种快速技术实现这一目的。我们的评估研究表明,手动快速调整至关重要,必须鼓励LMM提供不同长度的回答,特别是空回答套,真实/false问题是提高LMM所提出建议准确性的一个有用工具,LM的大小是一个关键因素,一个实体词典可以改进LM的得分。我们的评估研究表明,这些拟议技术可以大大提高最后预测的质量:ProP赢得LM-K BC竞赛的第2轨,以空白的回答方式,包括空回答套,真实/false 问题是提高LMBC/Hegreb/Meb的可用基准点。