Transferring knowledge across graphs plays a pivotal role in many high-stake domains, ranging from transportation networks to e-commerce networks, from neuroscience to finance. To date, the vast majority of existing works assume both source and target domains are sampled from a universal and stationary distribution. However, many real-world systems are intrinsically dynamic, where the underlying domains are evolving over time. To bridge the gap, we propose to shift the problem to the dynamic setting and ask: given the label-rich source graphs and the label-scarce target graphs observed in previous T timestamps, how can we effectively characterize the evolving domain discrepancy and optimize the generalization performance of the target domain at the incoming T+1 timestamp? To answer the question, for the first time, we propose a generalization bound under the setting of dynamic transfer learning across graphs, which implies the generalization performance is dominated by domain evolution and domain discrepancy between source and target domains. Inspired by the theoretical results, we propose a novel generic framework DyTrans to improve knowledge transferability across dynamic graphs. In particular, we start with a transformer-based temporal encoding module to model temporal information of the evolving domains; then, we further design a dynamic domain unification module to efficiently learn domain-invariant representations across the source and target domains. Finally, extensive experiments on various real-world datasets demonstrate the effectiveness of DyTrans in transferring knowledge from dynamic source domains to dynamic target domains.
翻译:跨图形之间的知识迁移在许多高风险领域中发挥着关键作用,包括交通运输网络、电子商务网络、神经科学和金融等领域。到目前为止,现有的大部分作品都假定源域和目标域都是从一个通用的、稳定的分布中抽样的。然而,许多现实世界的系统是本质上动态的,其中基础域随时间演变。为了弥补这一差距,我们提出了将问题转移到动态设置,并提出一个问题:在前T个时间戳中观察到的标签丰富的源图和标签稀疏的目标图中,如何有效地表征不断演变的域差异,并优化目标域在下一个T+1时间戳的泛化性能?为了回答这个问题,我们首次在动态图形之间的迁移学习设置下提出了一个泛化边界,它意味着泛化性能受到领域演变和源域与目标域之间的领域差异的支配。受到理论结果的启发,我们提出了一种新颖的通用框架DyTrans,以提高跨动态图形之间的知识可迁移性。特别是,我们从基于Transformer的时间编码模块开始,以模拟演变域的时间信息;然后,我们进一步设计了一个动态域统一模块,以有效地学习源域和目标域之间的域不变表示。最后,关于各种真实世界数据集的大量实验表明,DyTrans在将知识从动态源域转移到动态目标域方面具有显著的效果。