Sampling is a critical operation in Graph Neural Network (GNN) training that helps reduce the cost. Previous literature has explored improving sampling algorithms via mathematical and statistical methods. However, there is a gap between sampling algorithms and hardware. Without consideration of hardware, algorithm designers merely optimize sampling at the algorithm level, missing the great potential of promoting the efficiency of existing sampling algorithms by leveraging hardware features. In this paper, we pioneer to propose a unified programming model for mainstream sampling algorithms, termed GNNSampler, covering the critical processes of sampling algorithms in various categories. Second, to leverage the hardware feature, we choose the data locality as a case study, and explore the data locality among nodes and their neighbors in a graph to alleviate irregular memory access in sampling. Third, we implement locality-aware optimizations in GNNSampler for various sampling algorithms to optimize the general sampling process. Finally, we emphatically conduct experiments on large graph datasets to analyze the relevance among training time, accuracy, and hardware-level metrics. Extensive experiments show that our method is universal to mainstream sampling algorithms and helps significantly reduce the training time, especially in large-scale graphs.
翻译:在图形神经网络(GNN)培训中,抽样是有助于降低成本的关键操作。以前的文献探讨了通过数学和统计方法改进抽样算法的问题。然而,抽样算法和硬件之间存在差距。没有考虑硬件,算法设计师仅仅优化了算法层面的抽样,就失去了通过利用硬件功能提高现有抽样算法效率的巨大潜力。在本文中,我们先行提出主流抽样算法的统一编程模型,称为GNNSampler,涵盖不同类别取样算法的关键过程。第二,为了利用硬件特征,我们选择数据地点作为案例研究,并在图表中探讨节点及其邻居之间的数据地点,以缓解抽样中不规则的内存访问。第三,我们在GNNSampler对各种取样算法进行地点性优化,以优化一般取样过程。最后,我们重点在大图表数据集上进行实验,以分析培训时间、准确性和硬件级计量标准之间的关联性。广泛的实验表明,我们的方法是通用的,将抽样算法纳入主流,有助于大大减少培训时间,特别是在大比例图中。