Network slicing (NS) management devotes to providing various services to meet distinct requirements over the same physical communication infrastructure and allocating resources on demands. Considering a dense cellular network scenario that contains several NS over multiple base stations (BSs), it remains challenging to design a proper real-time inter-slice resource management strategy, so as to cope with frequent BS handover and satisfy the fluctuations of distinct service requirements. In this paper, we propose to formulate this challenge as a multi-agent reinforcement learning (MARL) problem in which each BS represents an agent. Then, we leverage graph attention network (GAT) to strengthen the temporal and spatial cooperation between agents. Furthermore, we incorporate GAT into deep reinforcement learning (DRL) and correspondingly design an intelligent real-time inter-slice resource management strategy. More specially, we testify the universal effectiveness of GAT for advancing DRL in the multi-agent system, by applying GAT on the top of both the value-based method deep Q-network (DQN) and a combination of policy-based and value-based method advantage actor-critic (A2C). Finally, we verify the superiority of the GAT-based MARL algorithms through extensive simulations.
翻译:网络切除(NS)管理致力于提供各种服务,以满足对同一有形通信基础设施的不同要求,并根据需求分配资源。考虑到包含多基站(BS)若干个NS的密集蜂窝网络情景,设计一个适当的实时虱子间资源管理战略仍然具有挑战性,以便应对频繁的BS交接和满足不同服务需求的波动。在本文件中,我们提议将这项挑战描述为一个多剂强化学习(MARL)问题,每个BS都代表一个代理。然后,我们利用图形关注网络加强代理商之间的时间和空间合作。此外,我们将GAT纳入深度强化学习(DRL),并相应地设计一个智能的实时虱子间资源管理战略。更具体地说,我们证明GAT在多试系统中推进DL的普遍有效性,在基于价值的方法的深Q网络(DQN)和基于政策和价值的方法优势的演员-critic(A2C)的组合中应用GAT(DAR),我们通过模拟GAT-AT的磁测算器来验证GAT的优势。