Promoting Coordination Through Electing First-moveAgent in Multi-Agent Reinforcement Learning

Learning to coordinate among multiple agents is an essential problem in multi-agent systems. Multi-agent reinforcement learning has long been a go-to tool in the complicated collaborative environment. However, most existing works are constrained by the assumption that all agents take actions simultaneously. In this paper, we endow the hierarchical order of play for the agents through electing a first-move agent and other agents take the best response to the first-move agent to obtain better coordination. We propose the algorithm EFA-DQN to implicitly model the coordination and learn the coordinated behavior in multi-agent systems. To verify the feasibility and demonstrate the effectiveness and efficiency of our algorithm, we conduct extensive experiments on several multi-agent tasks with different numbers of agents: Cooperative Navigation, Physical Deception, and The Google Football. The empirical results across the various scenarios show that our method achieves competitive advantages in terms of better performance and faster convergence, which demonstrates that our algorithm has broad prospects for addressing many complex real-world problems.