The literature in modern machine learning has only negative results for learning to communicate between competitive agents using standard RL. We introduce a modified sender-receiver game to study the spectrum of partially-competitive scenarios and show communication can indeed emerge in a competitive setting. We empirically demonstrate three key takeaways for future research. First, we show that communication is proportional to cooperation, and it can occur for partially competitive scenarios using standard learning algorithms. Second, we highlight the difference between communication and manipulation and extend previous metrics of communication to the competitive case. Third, we investigate the negotiation game where previous work failed to learn communication between independent agents (Cao et al., 2018). We show that, in this setting, both agents must benefit from communication for it to emerge; and, with a slight modification to the game, we demonstrate successful communication between competitive agents. We hope this work overturns misconceptions and inspires more research in competitive emergent communication.
翻译:现代机器学习的文献在学习使用标准RL的竞争性代理商之间交流方面只产生负面结果。 我们引入了修改的发送者-接收者游戏,以研究部分竞争情景的范围,并表明交流确实可以在竞争环境中出现。 我们从经验上为未来研究展示了三个关键利益。 首先,我们表明,交流与合作成比例,对于部分竞争情景而言,可以使用标准学习算法进行交流。 其次,我们强调沟通和操纵之间的区别,并将先前的通信标准扩展至竞争性案例。 第三,我们调查先前的工作未能学习独立代理商之间交流的谈判游戏(Cao et al.,2018年)。 我们表明,在这种环境下,两种代理商都必须从通信中获益,才能出现这种交流;在对游戏稍作修改后,我们展示了竞争性代理商之间的成功沟通。 我们希望这项工作能够推翻误解,并激励对竞争性新兴通信进行更多的研究。