Graph Neural Networks (GNNs) tend to suffer from high computation costs due to the exponentially increasing scale of graph data and the number of model parameters, which restricts their utility in practical applications. To this end, some recent works focus on sparsifying GNNs with the lottery ticket hypothesis (LTH) to reduce inference costs while maintaining performance levels. However, the LTH-based methods suffer from two major drawbacks: 1) they require exhaustive and iterative training of dense models, resulting in an extremely large training computation cost, and 2) they only trim graph structures and model parameters but ignore the node feature dimension, where significant redundancy exists. To overcome the above limitations, we propose a comprehensive graph gradual pruning framework termed CGP. This is achieved by designing a during-training graph pruning paradigm to dynamically prune GNNs within one training process. Unlike LTH-based methods, the proposed CGP approach requires no re-training, which significantly reduces the computation costs. Furthermore, we design a co-sparsifying strategy to comprehensively trim all three core elements of GNNs: graph structures, node features, and model parameters. Meanwhile, aiming at refining the pruning operation, we introduce a regrowth process into our CGP framework, in order to re-establish the pruned but important connections. The proposed CGP is evaluated by using a node classification task across 6 GNN architectures, including shallow models (GCN and GAT), shallow-but-deep-propagation models (SGC and APPNP), and deep models (GCNII and ResGCN), on a total of 14 real-world graph datasets, including large-scale graph datasets from the challenging Open Graph Benchmark. Experiments reveal that our proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of existing methods.
翻译:内建网络(GNNs)的计算成本往往很高,原因是图形数据规模和模型参数的急剧扩大,限制了其实际应用的实用性。为此,最近的一些工作侧重于用彩票假设(LTH)使 GNNs升级,以减少推论成本,同时保持性能水平。但是,基于LTH 的方法有两个重大缺陷:1)它们要求对密集模型进行详尽和迭接的培训,从而导致极高的培训计算成本;2)它们仅仅对深层的图形结构和模型参数进行细化,而忽略了存在重大冗余的节点特征。为了克服上述限制,我们提议了一个名为CGPG的全图渐进运行框架。这是通过在一个培训过程中设计一个在动态中将GNNNP的图形运行模式运行模式到动态的GNNNNPs, 与基于LTH的方法不同,拟议的CGP方法不需要再培训,这大大降低了计算成本。此外,我们设计了一个共同的阵列式和阵列式的阵列战略将GNNCs的所有三个核心要素都从GNS的图形结构结构结构中引入总的图式结构,其中包括CGPD的缩缩缩模型,将GPD的模型和模型,同时将GPOGOD的模型和模型将数据转换成一个新的模型,同时将GNPGPOPOD的模型,将我们在不断的流程的流程的流程中,将改进的流程的运行的运行中,将数据转换成的图。