This paper surveys the field of multiagent deep reinforcement learning. The combination of deep neural networks with reinforcement learning has gained increased traction in recent years and is slowly shifting the focus from single-agent to multiagent environments. Dealing with multiple agents is inherently more complex as (a) the future rewards depend on the joint actions of multiple players and (b) the computational complexity of functions increases. We present the most common multiagent problem representations and their main challenges, and identify five research areas that address one or more of these challenges: centralised training and decentralised execution, opponent modelling, communication, efficient coordination, and reward shaping. We find that many computational studies rely on unrealistic assumptions or are not generalisable to other settings; they struggle to overcome the curse of dimensionality or nonstationarity. Approaches from psychology and sociology capture promising relevant behaviours such as communication and coordination. We suggest that, for multiagent reinforcement learning to be successful, future research addresses these challenges with an interdisciplinary approach to open up new possibilities for more human-oriented solutions in multiagent reinforcement learning.
翻译:本文调查了多试剂深度强化学习的领域。深神经网络与强化学习的结合在最近几年中获得了更大的牵引力,并且正在慢慢地将重点从单一试剂环境转移到多剂环境。处理多种剂本身就更加复杂,因为(a) 未来的奖励取决于多个行为者的联合行动,(b) 功能的计算复杂性增加。我们提出了最常见的多剂问题表述及其主要挑战,并确定了解决其中一种或多种挑战的五个研究领域:集中培训和分散执行、对手建模、通信、高效协调和奖赏塑造。我们发现,许多计算研究依赖于不切实际的假设,或者无法向其他环境推广;它们努力克服维度或非常态的诅咒。心理学和社会学的方法抓住了沟通和协调等有希望的相关行为。我们建议,为了使多剂强化学习取得成功,今后的研究应当以多学科方法应对这些挑战,为多剂强化学习开辟新的可能性,在多剂强化学习中找到更面向人的解决方案。