Many real-world domains require safe decision making in the presence of uncertainty. In this work, we propose a deep reinforcement learning framework for approaching this important problem. We consider a risk-averse perspective towards model uncertainty through the use of coherent distortion risk measures, and we show that our formulation is equivalent to a distributionally robust safe reinforcement learning problem with robustness guarantees on performance and safety. We propose an efficient implementation that only requires access to a single training environment, and we demonstrate that our framework produces robust, safe performance on a variety of continuous control tasks with safety constraints in the Real-World Reinforcement Learning Suite.
翻译:许多现实世界领域都需要在不确定的情况下作出安全的决策。 在这项工作中,我们提出一个深入强化的学习框架,以解决这一重要问题。我们考虑通过使用连贯的扭曲风险措施,从风险角度来看待模型不确定性,我们表明,我们的提法相当于一个分布稳健的安全强化学习问题,保证业绩和安全的稳健性。我们建议一个有效的实施,只需要有一个单一的培训环境,我们表明,我们的框架在现实世界强化学习套件的安全限制下,在一系列连续控制任务上产生了有力和安全的业绩。