Reinforcement Learning and recently Deep Reinforcement Learning are popular methods for solving sequential decision making problems modeled as Markov Decision Processes. RL modeling of a problem and selecting algorithms and hyper-parameters require careful considerations as different configurations may entail completely different performances. These considerations are mainly the task of RL experts; however, RL is progressively becoming popular in other fields where the researchers and system designers are not RL experts. Besides, many modeling decisions, such as defining state and action space, size of batches and frequency of batch updating, and number of timesteps are typically made manually. For these reasons, automating different components of RL framework is of great importance and it has attracted much attention in recent years. Automated RL provides a framework in which different components of RL including MDP modeling, algorithm selection and hyper-parameter optimization are modeled and defined automatically. In this article, we explore the literature and present recent work that can be used in automated RL. Moreover, we discuss the challenges, open questions and research directions in AutoRL.
翻译:强化学习和最近深入强化学习是解决以Markov决定程序为模型的连续决策问题的流行方法。RL模拟问题和选择算法和超参数需要仔细考虑,因为不同的配置可能要求完全不同的性能。这些考虑主要是RL专家的任务;然而,RL在研究人员和系统设计者不是RL专家的其他领域越来越受欢迎。此外,许多模拟决定,例如界定状态和行动空间、批量规模和批次更新频率以及时间步骤数通常都是手工作出的。为此原因,自动生成RL框架的不同组成部分非常重要,近年来它引起了很大的注意。自动RL提供了一个框架,使RL的不同组成部分,包括MDP模型、算法选择和超参数优化能够自动建模和定义。在本条中,我们探讨了文献,介绍了可用于自动RL的近期工作。此外,我们讨论了AutoRL的挑战、开放问题和研究方向。