Mean-field games (MFG) were introduced to efficiently analyze approximate Nash equilibria in large population settings. In this work, we consider entropy-regularized mean-field games with a finite state-action space in a discrete time setting. We show that entropy regularization provides the necessary regularity conditions, that are lacking in the standard finite mean field games. Such regularity conditions enable us to design fixed-point iteration algorithms to find the unique mean-field equilibrium (MFE). Furthermore, the reference policy used in the regularization provides an extra means, through which one can control the behavior of the population. We first formulate the problem as a stochastic game with a large population of $N$ homogeneous agents. We establish conditions for the existence of a Nash equilibrium in the limiting case as $N$ tends to infinity, and we demonstrate that the Nash equilibrium for the infinite population case is also an $\epsilon$-Nash equilibrium for the $N$-agent regularized game, where the sub-optimality $\epsilon$ is of order $\mathcal{O}\big(1/\sqrt{N}\big)$. Finally, we verify the theoretical guarantees through a resource allocation example and demonstrate the efficacy of using a reference policy to control the behavior of a large population of agents.
翻译:引入了常规游戏(MFG)来有效分析大型人口环境中的近似 Nash 平均平衡。 在这项工作中, 我们考虑在离散的时间设置中, 使用有限的州行动空间, 使用有限的州行动空间, 使用不固定的普通游戏 。 这种常规性条件使我们能够设计固定点的重复算法, 以找到独特的平均平衡 。 此外, 正规化中使用的参考政策提供了一种额外手段, 通过这种手段, 一个人可以控制人口的行为 。 我们首先将问题发展成一个随机游戏, 与大量人口( 美元同质剂) 形成一个不固定的州行动空间 。 我们为限制情况下的纳什平衡创造了必要的常规性条件, 因为这些条件在标准限值平均游戏中是缺乏的。 我们证明, 无限人口案例中的纳什平衡也是一种美元和纳什的平衡, 用于美元试剂正规化的游戏, 在那里, 亚优度 $\ silon$ 可以控制人口行为 。