General intelligence requires solving tasks across many domains. Current reinforcement learning algorithms carry this potential but are held back by the resources and knowledge required to tune them for new tasks. We present DreamerV3, a general and scalable algorithm based on world models that outperforms previous approaches across a wide range of domains with fixed hyperparameters. These domains include continuous and discrete actions, visual and low-dimensional inputs, 2D and 3D worlds, different data budgets, reward frequencies, and reward scales. We observe favorable scaling properties of DreamerV3, with larger models directly translating to higher data-efficiency and final performance. Applied out of the box, DreamerV3 is the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula, a long-standing challenge in artificial intelligence. Our general algorithm makes reinforcement learning broadly applicable and allows scaling to hard decision making problems.
翻译:一般情报要求解决许多领域的任务。当前的强化学习算法具有这一潜力,但被调适新任务所需要的资源和知识所阻挡。我们展示了DreamerV3, 这是一种基于世界模型的通用且可扩展的算法,它以比以往在一系列广泛领域采用固定超参数的方法更优于以往方法的世界模型为基础。这些领域包括连续和分散的行动、视觉和低维输入、2D和3D世界、不同的数据预算、奖励频率和奖赏尺度。我们观察了DreamerV3的有利缩放特性,其较大的模型直接转换为更高的数据效率和最终性能。从盒子中应用,DreamerV3是第一个在没有人类数据或课程的情况下从零到零收集钻钻石的算法,这是人工智能的长期挑战。我们的一般算法使强化学习广泛适用,并允许扩大到困难的决策问题。