Dynamic Graph Neural Networks (DGNNs) have been broadly applied in various real-life applications, such as link prediction and pandemic forecast, to capture both static structural information and temporal characteristics from dynamic graphs. Combining both time-dependent and -independent components, DGNNs manifest substantial parallel computation and data reuse potentials, but suffer from severe memory access inefficiency and data transfer overhead under the canonical one-graph-at-a-time training pattern. To tackle the challenges, we propose PiPAD, a $\underline{\textbf{Pi}}pelined$ and $\underline{\textbf{PA}}rallel$ $\underline{\textbf{D}}GNN$ training framework for the end-to-end performance optimization on GPUs. From both the algorithm and runtime level, PiPAD holistically reconstructs the overall training paradigm from the data organization to computation manner. Capable of processing multiple graph snapshots in parallel, PiPAD eliminates the unnecessary data transmission and alleviates memory access inefficiency to improve the overall performance. Our evaluation across various datasets shows PiPAD achieves $1.22\times$-$9.57\times$ speedup over the state-of-the-art DGNN frameworks on three representative models.
翻译:动态图形神经网络(DGNNs)被广泛应用于各种实际应用,例如链接预测和大流行病预测,以从动态图形中捕捉静态结构信息和时间特性。DGNNs将基于时间的和独立的部分结合起来,显示大量的平行计算和数据再利用潜力,但面临严重的记忆存取效率低下和数据转移管理,而根据Canonical一线一线一时培训模式,数据转移管理费用严重不足。为了应对挑战,我们提议PIPAD,一个$underline textbf{PA_rallel$和$underline_textb{PA_rallel$\untline_textb{D_GNNN$在GPUs端端至终端业绩优化培训框架中。从算法和运行时,PPADAD整体地重建了从数据组织到计算方法的总体培训模式。能够同时处理多张图形的图像,PPPADAD消除不必要的数据传输,并减轻记忆效率,以提高总体业绩。我们对PPPPADD$的估价模型展示了三州的进度。