Mean Field Games (MFG) are the class of games with a very large number of agents and the standard equilibrium concept is a Mean Field Equilibrium (MFE). Algorithms for learning MFE in dynamic MFGs are unknown in general. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementarities (MFG-SC). We introduce a natural refinement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We also introduce a model-free and a model-based approach to learning T-MFE and provide sample complexities of both algorithms. We also develop a fully online learning scheme that obviates the need for a simulator. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications.
翻译:场中普通运动(MFG)是具有大量代理人的游戏类别,标准平衡概念是平均场平衡(MFE) 。 在动态MFG中学习 MFE 的算法一般并不为人所知。 我们的侧重点是拥有一种称为战略互补(MFG-SC)的单一属性的重要子类。 我们对我们称之为Trembling-Hand-Perfect MFE(T-MFE)的平衡概念进行了自然的完善,它允许代理人在计算这种随机化对其报酬的影响时采用一种随机化的尺度。 我们提出一种在已知模式下计算 T-MFE 的简单算法。 我们还采用了一种无模型和基于模型的方法来学习T-MFE 并提供两种算法的样本复杂性。 我们还开发了一种完全在线的学习计划,避免了模拟器的需要。 最后,我们通过真实世界应用的范例对拟议算法的绩效进行了经验评估。