Graph neural networks (GNNs) are deep learning models designed specifically for graph data, and they typically rely on node features as the input to the first layer. When applying such a type of network on the graph without node features, one can extract simple graph-based node features (e.g., number of degrees) or learn the input node representations (i.e., embeddings) when training the network. While the latter approach, which trains node embeddings, more likely leads to better performance, the number of parameters associated with the embeddings grows linearly with the number of nodes. It is therefore impractical to train the input node embeddings together with GNNs within graphics processing unit (GPU) memory in an end-to-end fashion when dealing with industrial-scale graph data. Inspired by the embedding compression methods developed for natural language processing (NLP) tasks, we develop a node embedding compression method where each node is compactly represented with a bit vector instead of a floating-point vector. The parameters utilized in the compression method can be trained together with GNNs. We show that the proposed node embedding compression method achieves superior performance compared to the alternatives.
翻译:图形神经网络( GNNS) 是专为图形数据设计的深层次学习模型, 通常以节点特性作为输入第一个层的输入。 在图形中应用这种类型的网络时, 没有节点特性, 人们可以提取简单的图形节点特征( 例如, 度数), 或者在培训网络时学习输入节点表示( 嵌入 ) 。 后一种方法, 即训练节点嵌入, 更有可能导致更好的性能, 与嵌入相关的参数数量随着节点数的数而线性增长。 因此, 在处理工业级图形数据时, 将输入节点与 GNNPS 一起嵌入在图形处理单元( GPU) 记忆中是不切实际的。 在为自然语言处理( NLP) 任务开发的嵌入压缩方法的启发下, 我们开发一种节点嵌入压缩方法, 即每个节点以小矢量代表一种小矢量, 而不是浮动点矢量。 因此, 压缩方法中使用的参数可以与 GNNNPS 一起训练。 我们显示高压方法 。