Reinforcement learning (RL) has proven its worth in a series of artificial domains, and is beginning to show some successes in real-world scenarios. However, much of the research advances in RL are hard to leverage in real-world systems due to a series of assumptions that are rarely satisfied in practice. In this work, we identify and formalize a series of independent challenges that embody the difficulties that must be addressed for RL to be commonly deployed in real-world systems. For each challenge, we define it formally in the context of a Markov Decision Process, analyze the effects of the challenge on state-of-the-art learning algorithms, and present some existing attempts at tackling it. We believe that an approach that addresses our set of proposed challenges would be readily deployable in a large number of real world problems. Our proposed challenges are implemented in a suite of continuous control environments called the realworldrl-suite which we propose an as an open-source benchmark.
翻译:强化学习(RL)在一系列人工领域证明了其价值,并开始在现实世界情景中表现出一些成功。然而,由于一系列在实践中很少满足的假设,该学习的很多研究进展很难在现实世界系统中发挥杠杆作用。在这项工作中,我们确定并正式确定一系列独立挑战,其中体现了必须解决的难题,使该学习在现实世界系统中普遍部署。对于每一项挑战,我们都在马可夫决策过程中正式界定了它,分析了挑战对最新学习算法的影响,并提出了应对它的一些现有尝试。我们认为,解决我们提出的一系列挑战的方法将很容易在大量现实世界问题中部署。我们提出的挑战是在一系列持续控制环境中实施的,称为“现实世界”,我们建议将其作为一个公开源基准。