Multicast Scheduling over Multiple Channels: A Distribution-Embedding Deep Reinforcement Learning Method

Multicasting is an efficient technique to simultaneously transmit common messages from the base station (BS) to multiple mobile users (MUs). The multicast scheduling problem over multiple channels, which jointly minimizes the energy consumption of the BS and the latency of serving asynchronized requests from the MUs, is formulated as an infinite-horizon Markov decision process (MDP) with large discrete action space, multiple time-varying constraints, and multiple time-invariant constraints, which has not been efficiently solved in the literatures. To address this problem, this paper proposes a novel algorithm called distribution-embedding multi-agent proximal policy optimization (DE-MAPPO), which consists of two parts: a modified MAPPO module and a distribution-embedding module. The former one modifies MAPPO's offline training and online applying mechanisms to handle the large discrete action space issue and time-varying constraints, and the latter one iteratively adjusts the action distribution to satisfy the time-invariant constraints. Moreover, as a benchmark, a performance upper bound of the considered MDP is derived by solving a two-step optimization problem. Numerical experiments show that the proposed algorithm achieves comparable performance to the derived benchmark in typical scenarios.