Graph convolutional networks have made great progress in graph-based semi-supervised learning. Existing methods mainly assume that nodes connected by graph edges are prone to have similar attributes and labels, so that the features smoothed by local graph structures can reveal the class similarities. However, there often exist mismatches between graph structures and labels in many real-world scenarios, where the structures may propagate misleading features or labels that eventually affect the model performance. In this paper, we propose a multi-task self-distillation framework that injects self-supervised learning and self-distillation into graph convolutional networks to separately address the mismatch problem from the structure side and the label side. First, we formulate a self-supervision pipeline based on pre-text tasks to capture different levels of similarities in graphs. The feature extraction process is encouraged to capture more complex proximity by jointly optimizing the pre-text task and the target task. Consequently, the local feature aggregations are improved from the structure side. Second, self-distillation uses soft labels of the model itself as additional supervision, which has similar effects as label smoothing. The knowledge from the classification pipeline and the self-supervision pipeline is collectively distilled to improve the generalization ability of the model from the label side. Experiment results show that the proposed method obtains remarkable performance gains under several classic graph convolutional architectures.
翻译:在基于图形的半监督的学习中,图形图变网络取得了巨大的进步。现有方法主要假设通过图形边缘连接的节点容易具有相似的属性和标签,因此由本地图形结构平滑的特征可以显示类类相似性。然而,在许多真实世界情景中,图形结构和标签之间往往存在不匹配的情况,这些结构可能会传播误导性特征或标签,最终影响模型性能。在本文件中,我们提议了一个多任务自省框架,将自监督性学习和自我蒸馏注入图形卷网络,以分别解决结构侧和标签侧的不匹配问题。首先,我们根据预文本任务设计了一个自我监督管道,以捕捉图形不同程度的相似性。鼓励特征提取过程通过共同优化预文本任务和目标任务来捕捉更复杂的邻近性。因此,从结构侧面改进了本地特征汇总。 其次,自我蒸馏使用软标签本身作为额外的监管,这具有类似效果,将平滑的图面结构与标签面和标签侧面分别标。首先,我们根据预版任务设计,设计出一个自我监督性的管道性化的管道性能,以集体化方法展示性业绩,从一些常规结构的模型获得。在常规化过程中的进度图状图状图上,从而展示了预化成果。