This paper addresses the problem of decentralized spectrum sharing in vehicle-to-everything (V2X) communication networks. The aim is to provide resource-efficient coexistence of vehicle-to-infrastructure(V2I) and vehicle-to-vehicle(V2V) links. A recent work on the topic proposes a multi-agent reinforcement learning (MARL) approach based on deep Q-learning, which leverages a fingerprint-based deep Q-network (DQN) architecture. This work considers an extension of this framework by combining Double Q-learning (via Double DQN) and transfer learning. The motivation behind is that Double Q-learning can alleviate the problem of overestimation of the action values present in conventional Q-learning, while transfer learning can leverage knowledge acquired by an expert model to accelerate learning in the MARL setting. The proposed algorithm is evaluated in a realistic V2X setting, with synthetic data generated based on a geometry-based propagation model that incorporates location-specific geographical descriptors of the simulated environment(outlines of buildings, foliage, and vehicles). The advantages of the proposed approach are demonstrated via numerical simulations.
翻译:本文探讨车辆到一切(V2X)通信网络中分散共享频谱的问题,目的是为车辆到基础设施(V2I)和车辆到车辆(V2V)连接提供资源效率高的共存;最近关于这一专题的工作提议了基于深层次Q学习的多剂强化学习(MARL)方法,该方法利用基于指纹的深层次Q网络(DQN)结构,利用基于指纹的深层次Q网络(DQN)结构。这项工作考虑将双Q学习(通过双QN)和转移学习结合起来,扩大这一框架。其背后的动机是,双Q学习能够减轻对常规Q学习中存在的行动价值的过高估计问题,而转让学习则能够利用专家模型获得的知识加速MARL环境中的学习。提议的算法是在现实的V2X设置中加以评价,根据基于地理测量的传播模型产生的合成数据,其中包括模拟环境(建筑、木质和车辆的外线)的特定地理描述器。拟议方法的优点通过数字模拟得到证明。