Knowledge graph embedding (KGE) that maps entities and relations into vector representations is essential for downstream tasks. Conventional KGE methods require relatively high-dimensional entity representations to preserve the structural information of knowledge graph, but lead to oversized model parameters. Recent methods reduce model parameters by adopting low-dimensional entity representations, while developing techniques (e.g., knowledge distillation) to compensate for the reduced dimension. However, such operations produce degraded model accuracy and limited reduction of model parameters. Specifically, we view the concatenation of all entity representations as an embedding layer, and then conventional KGE methods that adopt high-dimensional entity representations equal to enlarging the width of the embedding layer to gain expressiveness. To achieve parameter efficiency without sacrificing accuracy, we instead increase the depth and propose a deeper embedding network for entity representations, i.e., a narrow embedding layer and a multi-layer dimension lifting network (LiftNet). Experiments on three public datasets show that the proposed method (implemented based on TransE and DistMult) with 4-dimensional entity representations achieves more accurate link prediction results than counterpart parameter-efficient KGE methods and strong KGE baselines, including TransE and DistMult with 512-dimensional entity representations.
翻译:知识图谱嵌入(KGE)将实体和关系映射到向量表示的能力对下游任务至关重要。传统的KGE方法需要相对高维的实体表示以保留知识图谱的结构信息,但会导致过大的模型参数。最近的方法通过采用低维实体表示来减少模型参数,同时开发技术(如知识蒸馏)来弥补降维的影响。然而,这种操作会导致模型精度的降低和模型参数的减少受限。具体而言,我们将所有实体表示的串联视为一个嵌入层,而采用高维实体表示的传统KGE方法等效于扩展嵌入层的宽度以增强表现力。为了在不牺牲精度的情况下实现参数高效性,我们相反地增加了深度,并针对实体表示提出了一种更深的嵌入网络,即窄嵌入层和多层维度提升网络(LiftNet)。在三个公共数据集上的实验表明,基于TransE和DistMult实现的拟议方法:采用4维实体表示,比对应的参数高效KGE方法和强KGE基线(包括采用512维实体表示的TransE和DistMult)都能够获得更准确的链路预测结果。