Adversarial attacks on graphs have posed a major threat to the robustness of graph machine learning (GML) models. Naturally, there is an ever-escalating arms race between attackers and defenders. However, the strategies behind both sides are often not fairly compared under the same and realistic conditions. To bridge this gap, we present the Graph Robustness Benchmark (GRB) with the goal of providing a scalable, unified, modular, and reproducible evaluation for the adversarial robustness of GML models. GRB standardizes the process of attacks and defenses by 1) developing scalable and diverse datasets, 2) modularizing the attack and defense implementations, and 3) unifying the evaluation protocol in refined scenarios. By leveraging the GRB pipeline, the end-users can focus on the development of robust GML models with automated data processing and experimental evaluations. To support open and reproducible research on graph adversarial learning, GRB also hosts public leaderboards across different scenarios. As a starting point, we conduct extensive experiments to benchmark baseline techniques. GRB is open-source and welcomes contributions from the community. Datasets, codes, leaderboards are available at https://cogdl.ai/grb/home.
翻译:图表上的反向攻击对图形机学习模型的稳健性构成了重大威胁。自然,攻击者和捍卫者之间的军备竞赛日益加剧。然而,在同样和现实的条件下,对双方的战略往往没有进行公平的比较;为了缩小这一差距,我们提出了图表强力基准(GRB),目的是为GML模型的对抗性强健性提供一个可缩放、统一、模块化和可复制的评价。GRB将攻击和防御进程标准化,办法是:(1) 开发可缩放和多样化的数据集,(2) 模块化攻击和防御执行,(3) 将评价程序统一到经改进的情景中。通过利用GRB的管道,终端用户可以侧重于开发强大的GML模型,同时进行自动化数据处理和实验性评价。为了支持对图形对抗性学习进行公开和可复制的研究,GRB还在不同情景下设立了公共领导板。作为一个起点,我们进行了广泛的基准技术实验。GRBB是公开的来源,欢迎社区的贡献。Datagress/cobald. https. drops. datagrbs.