Knowledge graph (KG) link prediction is a fundamental task in artificial intelligence, with applications in natural language processing, information retrieval, and biomedicine. Recently, promising results have been achieved by leveraging cross-modal information in KGs, using ensembles that combine knowledge graph embeddings (KGEs) and contextual language models (LMs). However, existing ensembles are either (1) not consistently effective in terms of ranking accuracy gains or (2) impractically inefficient on larger datasets due to the combinatorial explosion problem of pairwise ranking with deep language models. In this paper, we propose a novel tiered ranking architecture CascadER to maintain the ranking accuracy of full ensembling while improving efficiency considerably. CascadER uses LMs to rerank the outputs of more efficient base KGEs, relying on an adaptive subset selection scheme aimed at invoking the LMs minimally while maximizing accuracy gain over the KGE. Extensive experiments demonstrate that CascadER improves MRR by up to 9 points over KGE baselines, setting new state-of-the-art performance on four benchmarks while improving efficiency by one or more orders of magnitude over competitive cross-modal baselines. Our empirical analyses reveal that diversity of models across modalities and preservation of individual models' confidence signals help explain the effectiveness of CascadER, and suggest promising directions for cross-modal cascaded architectures. Code and pretrained models are available at https://github.com/tsafavi/cascader.
翻译:知识图(KG)链接预测是人工智能的一项基本任务,在自然语言处理、信息检索和生物医学方面的应用。最近,通过利用KGs的跨模式信息,利用知识图嵌入(KGes)和背景语言模型(LMs)相结合的组合,实现了令人乐观的成果。然而,现有的组合:(1) 在精度增益排名方面,并非一贯有效,或者(2) 由于与深层语言模型相配的组合式爆炸性排名问题,大数据集上的效率不切实际。在本文中,我们提议了一个新的分级级结构卡卡塞德(Cascarder),以保持全套组合的准确性,同时大幅度提高效率。卡卡卡德(LMs)利用LMs重新排序更高效基础语言模型的输出,依靠一个适应性子集选择方案,目的是在尽可能降低精度增量的同时,在KGEEO上取得最大精度的精确度。 广泛实验表明,CacardER(MRRRRRRR)比现有基线高9点,在四种基准上设定新的状态,同时以一或更多水平来提高我们具有竞争力的SBIAL-SAL-Slation-Sleval-Slational-Slational-Slational-Slational-Slations laxxxxxxxxxxxxxxxxx