Topology impacts important network performance metrics, including link utilization, throughput and latency, and is of central importance to network operators. However, due to the combinatorial nature of network topology, it is extremely difficult to obtain an optimal solution, especially since topology planning in networks also often comes with management-specific constraints. As a result, local optimization with hand-tuned heuristic methods from human experts are often adopted in practice. Yet, heuristic methods cannot cover the global topology design space while taking into account constraints, and cannot guarantee to find good solutions. In this paper, we propose a novel deep reinforcement learning (DRL) algorithm, called Advantage Actor Critic-Graph Searching (A2C-GS), for network topology optimization. A2C-GS consists of three novel components, including a verifier to validate the correctness of a generated network topology, a graph neural network (GNN) to efficiently approximate topology rating, and a DRL actor layer to conduct a topology search. A2C-GS can efficiently search over large topology space and output topology with satisfying performance. We conduct a case study based on a real network scenario, and our experimental results demonstrate the superior performance of A2C-GS in terms of both efficiency and performance.
翻译:地形学影响到重要的网络性能衡量标准,包括连接利用、吞吐量和延缓度,对网络运营商至关重要,然而,由于网络地形学的组合性质,很难找到最佳解决办法,特别是网络的地形规划往往也伴随着管理上的限制,因此,往往在实践中采用由人类专家手工调制的体温方法,对地方进行优化;然而,在考虑制约因素的同时,超自然法方法无法覆盖全球地形设计空间,也无法保证找到良好的解决办法;在本文件中,我们提出一种新的深度强化学习(DRL)算法,称为Advantage Acentageor Critic-Graph搜索(A2C-GS),用于网络地形优化。A2C-GS由三个新的组成部分组成,包括验证所生成的网络地形学的正确性能,一个图形神经网络(GNNN),以及一个进行地形学搜索的DRL行为者层。A2-C可以有效地搜索大型顶层和产出表空间,称为A2C-A2S-A-S-S-S-PA-S-S-S-PL-S-S-S-PS-S-S-S-S-S-S-S-PL-S-S-S-S-PL-PL-S-SDS-S-S-S-S-S-PL-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-PS-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S