Graph neural networks (GNNs) have emerged as the standard method for numerous tasks on graph-structured data such as node classification. However, real-world graphs are often evolving over time and even new classes may arise. We model these challenges as an instance of lifelong learning, in which a learner faces a sequence of tasks and may take over knowledge acquired in past tasks. Such knowledge may be stored explicitly as historic data or implicitly within model parameters. In this work, we systematically analyze the influence of implicit and explicit knowledge. Therefore, we present an incremental training method for lifelong learning on graphs and introduce a new measure based on $k$-neighborhood time differences to address variances in the historic data. We apply our training method to five representative GNN architectures and evaluate them on three new lifelong node classification datasets. Our results show that no more than 50% of the GNN's receptive field is necessary to retain at least 95% accuracy compared to training over the complete history of the graph data. Furthermore, our experiments confirm that implicit knowledge becomes more important when fewer explicit knowledge is available.
翻译:图表神经网络(GNNs)已经成为图形结构数据(如节点分类)中众多任务的标准方法。然而,真实世界的图表往往随着时间而变化,甚至可能出现新的类别。我们把这些挑战作为终身学习的例子,学习者面临一系列任务,并可能接管过去任务中获得的知识。这些知识可以明确作为历史数据存储,或者隐含在模型参数中。在这项工作中,我们系统地分析隐含和明确知识的影响。因此,我们提出了一个在图表上进行终身学习的渐进培训方法,并引入基于美元近距离时间差异的新措施,以解决历史数据的差异。我们把我们的培训方法应用到五个具有代表性的GNN结构中,并用三个新的终身节点分类数据集来评估这些挑战。我们的结果显示,与对图表数据完整历史的培训相比,GNN的开放领域至少需要50%的准确度才能保留到至少95%。此外,我们的实验证实,如果掌握的清晰知识较少,隐含知识就更加重要了。