UNTER：一种用于增强预训练语言模型的统一知识接口 (UNTER: A Unified Knowledge Interface for Enhancing Pre-trained Language Models)

Recent research demonstrates that external knowledge injection can advance pre-trained language models (PLMs) in a variety of downstream NLP tasks. However, existing knowledge injection methods are either applicable to structured knowledge or unstructured knowledge, lacking a unified usage. In this paper, we propose a UNified knowledge inTERface, UNTER, to provide a unified perspective to exploit both structured knowledge and unstructured knowledge. In UNTER, we adopt the decoder as a unified knowledge interface, aligning span representations obtained from the encoder with their corresponding knowledge. This approach enables the encoder to uniformly invoke span-related knowledge from its parameters for downstream applications. Experimental results show that, with both forms of knowledge injected, UNTER gains continuous improvements on a series of knowledge-driven NLP tasks, including entity typing, named entity recognition and relation extraction, especially in low-resource scenarios.

翻译：最近的研究表明，外部知识注入可以在各种下游NLP任务中推进预训练的语言模型（PLMs）。然而，现有的知识注入方法既适用于结构化知识，也适用于非结构化知识，缺乏统一的使用方式。在本文中，我们提出了一种统一知识接口（UNTER），以提供统一的视角来利用结构化知识和非结构化知识。在UNTER中，我们采用解码器作为统一知识接口，将编码器中获得的跨度表示与它们对应的知识进行对齐。这种方法使得编码器能够从其参数中统一调用与跨度相关的知识，以用于下游应用。实验结果表明，使用注入的两种形式的知识，UNTER在一系列的知识驱动NLP任务上实现持续的改进，包括实体类型化，命名实体识别和关系抽取，特别是在低资源场景中。