The recent growth of emergent network applications (e.g., satellite networks, vehicular networks) is increasing the complexity of managing modern communication networks. As a result, the community proposed the Digital Twin Networks (DTN) as a key enabler of efficient network management. Network operators can leverage the DTN to perform different optimization tasks (e.g., Traffic Engineering, Network Planning). Deep Reinforcement Learning (DRL) showed a high performance when applied to solve network optimization problems. In the context of DTN, DRL can be leveraged to solve optimization problems without directly impacting the real-world network behavior. However, DRL scales poorly with the problem size and complexity. In this paper, we explore the use of Evolutionary Strategies (ES) to train DRL agents for solving a routing optimization problem. The experimental results show that ES achieved a training time speed-up of 128 and 6 for the NSFNET and GEANT2 topologies respectively.
翻译:最近新兴网络应用(如卫星网络、车辆网络)的增长使现代通信网络管理的复杂性日益复杂,因此,社区提议数字双网络(DTN)作为高效网络管理的关键促进因素;网络运营商可以利用DTN执行不同的优化任务(如交通工程、网络规划);深度强化学习(DRL)在用于解决网络优化问题时表现良好;在DTN方面,DRL可以被用来解决优化问题,而不会直接影响现实世界网络行为;然而,DRL与问题的规模和复杂性不相称;在本文件中,我们探索如何利用进化战略(ES)培训DRL代理商解决路线优化问题;实验结果显示,ES在NSFNET和GENT2地形学方面,培训速度分别达到128和6。