Temporal relation prediction in incomplete temporal knowledge graphs (TKGs) is a popular temporal knowledge graph completion (TKGC) problem in both transductive and inductive settings. Traditional embedding-based TKGC models (TKGE) rely on structured connections and can only handle a fixed set of entities, i.e., the transductive setting. In the inductive setting where test TKGs contain emerging entities, the latest methods are based on symbolic rules or pre-trained language models (PLMs). However, they suffer from being inflexible and not time-specific, respectively. In this work, we extend the fully-inductive setting, where entities in the training and test sets are totally disjoint, into TKGs and take a further step towards a more flexible and time-sensitive temporal relation prediction approach SST-BERT, incorporating Structured Sentences with Time-enhanced BERT. Our model can obtain the entity history and implicitly learn rules in the semantic space by encoding structured sentences, solving the problem of inflexibility. We propose to use a time masking MLM task to pre-train BERT in a corpus rich in temporal tokens specially generated for TKGs, enhancing the time sensitivity of SST-BERT. To compute the probability of occurrence of a target quadruple, we aggregate all its structured sentences from both temporal and semantic perspectives into a score. Experiments on the transductive datasets and newly generated fully-inductive benchmarks show that SST-BERT successfully improves over state-of-the-art baselines.
翻译:在不完整的时间知识图谱(TKGs)中进行时间关系预测是目前研究的热点,包括归纳和传导两种情形的时间知识图谱完备问题(TKGC)。传统的基于嵌入的TKGC模型(TKGE)依靠结构化连接,仅能处理固定的实体集合,即传导式的情况下。而在归纳的情况下,测试TKG包含新出现的实体,最新的方法则基于符号规则或预训练语言模型(PLM)。然而,它们分别存在不灵活性和无法针对时间的问题。本文将全归纳设定进一步扩展到TKG中,并提出了一种更加灵活和时间敏感的时间关系预测方法SST-BERT,将结构化句子和时间增强BERT相结合。我们的模型通过编码结构化句子可以获得实体历史信息,并在语义空间中隐式地学习规则,从而解决了不灵活性的问题。我们提出采用时间掩蔽的MLM任务,通过在专为TKG生成的富含时间令牌的语料库中预训练BERT,增强了SST-BERT的时间感知性。为了计算目标四元组出现的概率,我们将其所有的结构化句子从时间和语义两个方面进行汇总得到一个分数。在传导式数据集和新生成的全归纳式基准上的实验表明,SST-BERT成功地提高了对最先进的基线模型的预测效果。