Age-minimal Multicast by Graph Attention Reinforcement Learning

Age of Information (AoI) has emerged as a prominent metric for evaluating the timeliness of information in time-critical applications. Applications, including video streaming, virtual reality, and metaverse platforms, necessitate the use of multicast communication. Optimizing AoI in multicast networks is challenging due to the coupled multicast routing and scheduling decisions, the network dynamics, and the complexity of the multicast. This paper focuses on dynamic multicast networks and aims to minimize the expected average AoI through the integration of multicast routing and scheduling. To address the inherent complexity of the problem, we first propose to apply reinforcement learning (RL) to learn the heuristics of multicast routing, based on which we decompose the original problem into two subtasks that are amenable to hierarchical RL methods. Subsequently, we propose an innovative framework based on graph attention networks (GATs) and prove its contraction mapping property. Such a GAT framework effectively captures graph information used in the hierarchical RL framework with superior generalization capabilities. To validate our framework, we conduct experiments on three datasets, including a real-world dataset called AS-733, and show that our proposed scheme reduces the average weighted AoI by $38.2\%$ and the weighted peak age by $43.4\%$ compared to baselines over all datasets in dynamic networks.