Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. However, none of them provide theoretical insights into the design of their frameworks, or clear requirements and guarantees towards their transferability. In this work, we establish a theoretically grounded and practically useful framework for the transfer learning of GNNs. Firstly, we propose a novel view towards the essential graph information and advocate the capturing of it as the goal of transferable GNN training, which motivates the design of EGI (Ego-Graph Information maximization) to analytically achieve this goal. Secondly, when node features are structure-relevant, we conduct an analysis of EGI transferability regarding the difference between the local graph Laplacians of the source and target graphs. We conduct controlled synthetic experiments to directly justify our theoretical conclusions. Comprehensive experiments on two real-world network datasets show consistent results in the analyzed setting of direct-transfering, while those on large-scale knowledge graphs show promising results in the more practical setting of transfering with fine-tuning.
翻译:在各种应用中,图形神经网络(GNNs)取得了优异的绩效,但培训专门的GNNs对于大型图表来说可能成本很高。最近的一些工作开始研究GNS的预培训。然而,没有一项工作对框架的设计提供理论洞察,或对其可转让性提出明确的要求和保证。在这项工作中,我们为GNS的转移学习建立了一个具有理论基础和实际实用价值的框架。首先,我们对基本图形信息提出了一种新颖的观点,并倡导将它作为可转让GNN培训的目标加以捕捉,从而推动设计EGI(Ego-Graph信息最大化)来分析实现这一目标。第二,当节点特征与结构相关时,我们分析EGI在源和目标图的本地图形Laplaces之间差异方面的可转让性。我们进行有控制的合成实验,以直接证明我们的理论结论是正确的。关于两个真实世界网络数据集的全面实验显示直接转让的分析结果一致,而关于大规模知识图表的实验则显示在更切合实际的转换过程中取得有希望的结果。