力荐! Xavier Bresson的GNN在线讲座1-6

图神经网络（Graph Neural Networks）系列讲座

报告专家：Xavier Bresson教授（科技部高端外专项目20200027095资助）

邀请人：袁景 教授（西安电子科技大学）

主持人：唐晓颖 教授（南方科技大学）

              王珊珊 研究员（深圳先进研究院）

报告时间：2021/12/13-17，2021/12/20，16:30-17:30（北京时间）

Zoom会议：

https://nus-sg.zoom.us/j/85913613676?pwd=UzRScU9HQ2tiWEF6QVp0OHpXdWF2UT09

会议 ID：859 1361 3676

Password：333867

报告人简介

Xavier Bresson教授就职于新加坡国立大学计算机系，在结合了现代图论和神经网络技术的图神经网络（GNN）方法研究方面做出突出贡献。2016年，Xavier Bresson教授获得新加坡最大单笔科研基金NRF资助。他同时是图神经网络研究领域引用率最高的一篇文章作者，并被Yoshua Bengio引用作为对比图神经网络算法的框架基准之一。最近几年，他组织了美国纯数学与应用数学研究所IPAM每年的“深度学习与组合优化”研讨会，并受邀于专业顶会NIPS’17, CVPR’17以及SIAM’18做“基于图与流形的几何深度学习”讲座，在顶会 KDD’21, AAAI’21与ICML’20的“几何表征学习”研讨会、ICLR’20的“深度神经模型与微分方程”做相关邀请报告。受邀在美国纽约大学Yann LeCun的讲授课程中作为特约讲师。

讲座安排

1. Graph Neural Networks – Part 1 : Spectral Techniques     12/13/2021
2. Graph Neural Networks – Part 2 : Spatial Techniques     12/14/2021
3. Benchmarking Graph Neural Networks       12/06/2021
4. Graph Neural Networks with Learnable Structural and Positional Representations  12/09/2021
5. Graph Convolutional Neural Networks for Molecule Generation    12/13/2021
6. Learning to Solve the Traveling Salesman Problem      12/16/2021

讲座摘要

Talk 1 Graph Neural Networks – Part 1 : Spectral Techniques 

Date : 13 Dec 2021, 4:30-5:30 pm 

Abstract : In the past years, deep learning methods have achieved unprecedented performance on a broad range of problems in various fields from computer vision to speech recognition. So far research has mainly focused on developing deep learning methods for grid-structured data, while many important applications have to deal with graph-structured data. Such geometric data are becoming increasingly important in computer graphics and 3D vision, sensor networks, drug design, biomedicine, recommendation systems, NLP and computer vision with knowledge graphs, and web applications. The purpose of this talk is to introduce convolutional neural networks architectures on graphs from a spectral perspective.

Talk 2  Graph Neural Networks – Part 2 : Spatial Techniques 

Date : 14 Dec 2021, 4:30-5:30 pm

Abstract : This is the second part of the introduction to Graph Neural Networks. In this talk, we will introduce the design of GNNs from a spatial perspective. We will also present one Python notebook demo of GNNs to help the participant to get started with this class of deep learning techniques. Finally, we will cover the extensive applications of GNNs in research and industry.

Talk 3 Benchmarking Graph Neural Networks 

Date : 15 Dec 2021, 4:30-5:30 pm 

Abstract : Graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. As the field grows, it becomes critical to identify key architectures and validate new ideas that generalize to larger, more complex datasets. Unfortunately, it has been increasingly difficult to gauge the effectiveness of new models in the absence of a standardized benchmark with consistent experimental settings. In this work, we introduce a reproducible GNN benchmarking framework, with the facility for researchers to add new models conveniently for arbitrary datasets. We demonstrate the usefulness of our framework by presenting a principled investigation into the recent Weisfeiler-Lehman GNNs (WL-GNNs) compared to message passing-based graph convolutional networks (GCNs) for a variety of graph tasks with medium-scale datasets.

Talk 4  Graph Neural Networks with Learnable Structural and Positional Representations 

Date : 16 Dec 2021, 4:30-5:30 pm 

Abstract : Graph neural networks have become the standard toolkit for analyzing and learning from data on graphs. GNNs have been applied to several domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural language processing. A major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish e.g. isomorphic nodes and other graph symmetries. An approach to tackle this issue is to introduce positional encoding (PE) of nodes, and inject it into the input layer, like in Transformer. Possible graph PE are graph Laplacian eigenvectors, but their sign is not uniquely defined. In this work, we propose to decouple structural and positional representation to make easy for the network to learn these two properties. We show that any GNN can actually be augmented by learnable PE, and improve the performance. We investigate sparse and fully-connected/transformer-like GNNs, and observe the usefulness to learn PE for both classes.

Talk 5 Graph Convolutional Neural Networks for Molecule Generation 

Date : 17 Dec 2021, 4:30-5:30 pm 

Abstract : A fundamental problem in drug discovery and material science is to design molecules with arbitrary optimized chemical properties. This is a highly challenging mathematical and computational problem. Molecules are intrinsically combinatorial. Any small perturbation in the chemical structure may result in large variation in the desired molecular property. Besides, the space of valid molecules is huge as the number of combinatorial permutations of atoms and bonds grows factorially. A common example is the space of drug-like molecules that is estimated between 10^23 and 10^60. Currently, most drugs are hand-crafting by years of trial-and-error by human experts in chemistry and pharmacology. The recent advances of machine learning and deep learning has opened a new research direction, with the promise to learn these molecular spaces for optimized molecule generation without hand-crafting them. In this talk, I will discuss a graph convolutional neural network architecture for the molecule generation task. The proposed approach consists of two steps. First, a graph ConvNet is used to auto-encode molecules in one-shot. Second, beam search is applied to the output of neural networks to produce a valid chemical solution. Numerical experiments demonstrate the performances of this learning system.

Talk 6  Learning to Solve the Traveling Salesman Problem 

Date : 20 Dec 2021, 4:30-5:30 pm 

Abstract : The Traveling Salesman Problem (TSP) is the most popular and most studied combinatorial problem, starting with von Neumann in 1951. It has driven the discovery of several optimization techniques such as cutting planes, branch-and-bound, local search, Lagrangian relaxation, and simulated annealing. The last five years have seen the emergence of promising techniques where (graph) neural networks have been capable to learn new combinatorial algorithms. The main question is whether deep learning can learn better heuristics from data, i.e. replacing human-engineered heuristics? This is appealing because developing algorithms to tackle NP-hard problems may require years of research, and many industry problems are combinatorial by nature. In this project, we propose to adapt the recent successful Transformer architecture originally developed for natural language processing to the combinatorial TSP. Training is done by reinforcement learning, hence without TSP training solutions, and decoding uses beam search. We report improved performances over recent learned heuristics.