Language models (LMs) are sentence-completion engines trained on massive corpora. LMs have emerged as a significant breakthrough in natural-language processing, providing capabilities that go far beyond sentence completion including question answering, summarization, and natural-language inference. While many of these capabilities have potential application to cognitive systems, exploiting language models as a source of task knowledge, especially for task learning, offers significant, near-term benefits. We introduce language models and the various tasks to which they have been applied and then review methods of knowledge extraction from language models. The resulting analysis outlines both the challenges and opportunities for using language models as a new knowledge source for cognitive systems. It also identifies possible ways to improve knowledge extraction from language models using the capabilities provided by cognitive systems. Central to success will be the ability of a cognitive agent to itself learn an abstract model of the knowledge implicit in the LM as well as methods to extract high-quality knowledge effectively and efficiently. To illustrate, we introduce a hypothetical robot agent and describe how language models could extend its task knowledge and improve its performance and the kinds of knowledge and methods the agent can use to exploit the knowledge within a language model.
翻译:语言模型(LMS)是受过大规模集体教学培训的完成判决的引擎。LMS已成为自然语言处理方面的一个重大突破,提供了远远超出完成刑期范围的能力,包括回答问题、总结和自然语言推断。虽然其中许多能力都有可能应用于认知系统,但利用语言模型作为任务知识的来源,特别是用于任务学习,可以带来重大、近期的好处。我们引入语言模型及其应用的各种任务,然后审查从语言模型中提取知识的方法。由此产生的分析概述了使用语言模型作为认知系统新知识来源的挑战和机遇。它还确定了利用认知系统提供的能力改进从语言模型中提取知识的可能方法。成功的核心将是认知代理人自己学习LM系统所隐含知识的抽象模型的能力,以及有效和高效地获取高质量知识的方法。我们引入一个假设机器人代理,并描述语言模型如何扩展其任务知识,改进其业绩,以及该代理人在语言模型内利用知识的种类和方法。