The problem of cooperation is of fundamental importance for human societies, with examples ranging from navigating road junctions to negotiating climate treaties. As the use of AI becomes more pervasive within society, the need for socially intelligent agents that are able to navigate these complex dilemmas is becoming increasingly evident. Direct punishment is an ubiquitous social mechanism that has been shown to benefit the emergence of cooperation within the natural world, however no prior work has investigated its impact on populations of learning agents. Moreover, although the use of all forms of punishment in the natural world is strongly coupled with partner selection and reputation, no existing work has provided a holistic analysis of their combination within multi-agent systems. In this paper, we present a comprehensive analysis of the behaviors and learning dynamics associated with direct punishment in multi-agent reinforcement learning systems and how this compares to third-party punishment, when both forms of punishment are combined with other social mechanisms such as partner selection and reputation. We provide an extensive and systematic evaluation of the impact of these key mechanisms on the emergence of cooperation. Finally, we discuss the implications of the use of these mechanisms in the design of cooperative AI systems.
翻译:合作问题是人类社会的根本问题,其实例从路口通路到气候条约谈判等,对人类社会具有根本重要性。随着AI的使用在社会上越来越普遍,对能够克服这些复杂困境的社会智能分子的需要越来越明显。直接惩罚是一个无处不在的社会机制,它证明有利于自然界内合作的出现,然而,以前的工作没有调查它对学习者人口的影响。此外,尽管在自然界使用各种形式的惩罚与伙伴的选择和声誉密切相关,但现有的工作没有提供对多试剂系统内这些惩罚组合的全面分析。在本文件中,我们全面分析与多试剂强化学习系统中的直接惩罚有关的行为和学习动态,以及当这两种惩罚形式与伙伴选择和声誉等其他社会机制相结合时,这与第三方惩罚如何相提并论。我们对这些关键机制对合作产生的影响进行了广泛和系统的评估。我们讨论了在设计合作性情报系统时使用这些机制的影响。我们讨论了在设计合作性情报系统时使用这些机制的情况。