Progress in reinforcement learning (RL) research is often driven by the design of new, challenging environments -- a costly undertaking requiring skills orthogonal to that of a typical machine learning researcher. The complexity of environment development has only increased with the rise of procedural-content generation (PCG) as the prevailing paradigm for producing varied environments capable of testing the robustness and generalization of RL agents. Moreover, existing environments often require complex build processes, making reproducing results difficult. To address these issues, we introduce GriddlyJS, a web-based Integrated Development Environment (IDE) based on the Griddly engine. GriddlyJS allows researchers to visually design and debug arbitrary, complex PCG grid-world environments using a convenient graphical interface, as well as visualize, evaluate, and record the performance of trained agent models. By connecting the RL workflow to the advanced functionality enabled by modern web standards, GriddlyJS allows publishing interactive agent-environment demos that reproduce experimental results directly to the web. To demonstrate the versatility of GriddlyJS, we use it to quickly develop a complex compositional puzzle-solving environment alongside arbitrary human-designed environment configurations and their solutions for use in automatic curriculum learning and offline RL. The GriddlyJS IDE is open source and freely available at \url{https://griddly.ai}.
翻译:强化学习(RL)研究的进展往往是由设计新的、具有挑战性的环境驱动的。 设计新的、具有挑战性的环境往往是一种代价高昂的工作,需要技能与典型的机器学习研究者相同。随着程序内容生成(PCG)的兴起,环境发展的复杂性随着程序内容生成(PCG)的兴起而增加,成为生产能够测试RL代理商的稳健性和普遍性的各种环境的流行范例。此外,现有环境往往需要复杂的构建过程,从而难以复制成果。为了解决这些问题,我们引入了基于网络引擎的网格化JS(IDE),这是一个基于网络的综合开发环境。GridddlyJS允许研究人员使用方便的图形界面进行视觉设计和调试任意、复杂的PCG网格-world环境,以及可视觉化、评估和记录经过培训的代理商模型的性能。通过将RL工作流程与现代网络标准所促成的高级功能联系起来,GredlyJS允许将实验结果直接复制到网上的交互式代理商-环境演示。为了展示GridlyJS的多功能性,我们利用它来迅速开发复杂的拼图解环境,同时在任意的人类设计的智能环境中学习。