Alchemy is a new meta-learning environment rich enough to contain interesting abstractions, yet simple enough to make fine-grained analysis tractable. Further, Alchemy provides an optional symbolic interface that enables meta-RL research without a large compute budget. In this work, we take the first steps toward using Symbolic Alchemy to identify design choices that enable deep-RL agents to learn various types of abstraction. Then, using a variety of behavioral and introspective analyses we investigate how our trained agents use and represent abstract task variables, and find intriguing connections to the neuroscience of abstraction. We conclude by discussing the next steps for using meta-RL and Alchemy to better understand the representation of abstract variables in the brain.
翻译:炼金术是一种新的元学习环境,它丰富到足以包含有趣的抽象学,但又简单到可以进行精细分析。此外,炼金术提供了一个可选的符号界面,使得元RL研究能够在没有大量计算预算的情况下进行。在这项工作中,我们迈出了第一步,使用符号性炼金术来确定设计选择,使深RL代理能够学习各种抽象学。然后,利用各种行为和反省分析,我们调查我们受过训练的代理方如何使用和代表抽象任务变量,并找到与抽象神经科学的有趣联系。我们最后通过讨论使用元RL和炼金术的下一步步骤,以更好地了解大脑中抽象变量的表述。