The reinforcement learning (RL) research area is very active, with an important number of new contributions; especially considering the emergent field of deep RL (DRL). However a number of scientific and technical challenges still need to be resolved, amongst which we can mention the ability to abstract actions or the difficulty to explore the environment in sparse-reward settings which can be addressed by intrinsic motivation (IM). We propose to survey these research works through a new taxonomy based on information theory: we computationally revisit the notions of surprise, novelty and skill learning. This allows us to identify advantages and disadvantages of methods and exhibit current outlooks of research. Our analysis suggests that novelty and surprise can assist the building of a hierarchy of transferable skills that further abstracts the environment and makes the exploration process more robust.
翻译:强化学习(RL)研究领域非常活跃,有大量新的贡献,特别是考虑到深层RL(DRL)的新兴领域。然而,仍有一些科学和技术挑战需要解决,其中我们可以提到抽象行动的能力,或难以在少有回报的环境中探索环境,而这种环境可以通过内在动机来解决(IM)。 我们提议根据信息理论,通过一个新的分类学来对这些研究工作进行调查:我们计算出出出出出奇、新颖和技能学习的概念。这使我们能够找出方法的优缺点,并展示当前的研究前景。我们的分析表明,新颖和出奇不奇有助于建立可转让技能的等级,进一步总结环境,使勘探进程更加有力。