As modern games continue growing both in size and complexity, it has become more challenging to ensure that all the relevant content is tested and that any potential issue is properly identified and fixed. Attempting to maximize testing coverage using only human participants, however, results in a tedious and hard to orchestrate process which normally slows down the development cycle. Complementing playtesting via autonomous agents has shown great promise accelerating and simplifying this process. This paper addresses the problem of automatically exploring and testing a given scenario using reinforcement learning agents trained to maximize game state coverage. Each of these agents is rewarded based on the novelty of its actions, thus encouraging a curious and exploratory behaviour on a complex 3D scenario where previously proposed exploration techniques perform poorly. The curious agents are able to learn the complex navigation mechanics required to reach the different areas around the map, thus providing the necessary data to identify potential issues. Moreover, the paper also explores different visualization strategies and evaluates how to make better use of the collected data to drive design decisions and to recognize possible problems and oversights.
翻译:由于现代游戏在规模和复杂性方面都在继续增长,确保所有相关内容都经过测试,任何潜在问题都得到适当的识别和固定,就更具挑战性了。然而,仅仅使用人类参与者就试图最大限度地扩大测试范围,结果是一个乏味和难以安排的过程,通常会放慢开发周期。通过自主代理器进行的补充游戏测试显示极有希望加速和简化这一过程。本文件探讨了利用受过训练的强化学习代理器自动探索和测试某一场景的问题。每个这些代理商都因其行动的新颖性而得到奖励,从而鼓励在复杂的3D情景上采取好奇和探索性的行为,而先前提议的勘探技术在这种情景上表现不佳。好奇的代理商能够学习到地图周围不同区域所需的复杂的导航机械,从而提供必要的数据来查明潜在的问题。此外,本文还探讨了不同的视觉化战略,并评估如何更好地利用所收集的数据来推动设计决策,并发现可能存在的问题和监督。