Cooperative Multi-Agent Deep Reinforcement Learning for Reliable and Energy-Efficient Mobile Access via Multi-UAV Control

This paper addresses a novel multi-agent deep reinforcement learning (MADRL)-based multiple unmanned aerial vehicles (UAV) positioning algorithm for reliable mobile access services (i.e., UAVs work as mobile base stations), where the MADRL is designed by the concept of centralized training and distributed execution (CTDE) for multi-agent cooperation and coordination. The reliable mobile access services can be achieved in following two ways, i.e., (i) energy-efficient UAV operation and (ii) reliable wireless communication services. For energy-efficient UAV operation, the reward of our proposed MADRL algorithm contains the features for UAV energy consumption models in order to realize efficient operations. Furthermore, for reliable wireless communication services, the quality of service (QoS) requirements of individual users are considered as a part of rewards and 60GHz mmWave radio is used for mobile access. This paper considers the 60GHz mmWave access for utilizing the benefits of (i) ultra-wide-bandwidth for multi-Gbps high-speed communications and (ii) high-directional communications for spatial reuse that is obviously good for densely deployed users. Lastly, the performance of our proposed MADRL-based multi-UAV positioning algorithm is evaluated; and it can be confirmed that the proposed algorithm outperforms the other existing algorithms.