We study optimal strategies in two-player stochastic games that are played on a finite graph, equipped with a general payoff function. The existence of optimal strategies that do not make use of neither memory nor randomisation is a desirable property that vastly simplifies the algorithmic analysis of such games. Our main theorem gives a sufficient condition for the maximizer to possess such a simple optimal strategy. The condition is imposed on the payoff function, saying the payoff does not depend on any finite prefix (shift-invariant) and combining two trajectories does not give higher payoff than the payoff of the parts (submixing). The core technical property that enables the proof of the main theorem is that of the existence of epsilon-subgame-perfect strategies when the payoff function is shift-invariant. Furthermore, the same techniques can be used to prove a finite-memory transfer-type theorem: namely that for shift-invariant and submixing payoff functions, the existence of optimal finite-memory strategies in one-player games for the minimizer implies the existence of the same in two-player games. We show that numerous classical payoff functions are submixing and shift-invariant.
翻译:我们研究双玩者随机游戏的最佳策略,这些游戏在限定的图形上播放,配有一般报酬功能。存在不使用内存和随机化的最佳策略是一种可取的属性,可以大大简化这种游戏的算法分析。我们的主要理论为最大玩者拥有这种简单最佳策略提供了充分的条件。这个条件被强加在支付功能上,指出支付并不取决于任何有限的前缀(临时变换)和两种轨迹的组合不会带来高于部分(子混合)报酬的回报。使主要理论得到证明的核心技术属性是,当支付功能是变换-变换时,就存在埃普西朗-次游戏的超成功策略。此外,同样的技术可以用来证明一个有限的移动式转移类型理论:即变换变换和子组合支付功能,一个游戏中存在最优的定调策略,而一个游戏的最小变换功能是最小化的。