Active inference is a state-of-the-art framework for modelling the brain that explains a wide range of mechanisms such as habit formation, dopaminergic discharge and curiosity. However, recent implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to very good results in two different tasks. Additionally, Champion et al. (2021a) proposed a tree search approach based on structure learning. This was enabled by the development of a variational message passing approach to active inference (Champion et al., 2021b), which enables compositional construction of Bayesian networks for active inference. However, this message passing tree search approach, which we call branching-time active inference (BTAI), has never been tested empirically. In this paper, we present an experimental study of the approach (Champion et al., 2021a) in the context of a maze solving agent. In this context, we show that both improved prior preferences and deeper search help mitigate the vulnerability to local minima. Then, we compare BTAI to standard active inference (AI) on a graph navigation task. We show that for small graphs, both BTAI and AI successfully solve the task. For larger graphs, AI exhibits an exponential (space) complexity class, making the approach intractable. However, BTAI explores the space of policies more efficiently, successfully scaling to larger graphs.
翻译:活跃的推论是模拟大脑模型的最先进的框架,它解释了多种机制,如习惯形成、多巴胺释放和好奇心等。然而,在计算直至时间范围之前所有可能的政策之前的先前政策时,最近的执行受到指数(空间和时间)复杂等级的影响。Fountas等人(202020年)利用蒙特卡洛树搜索来解决这个问题,导致在两个不同的任务中取得非常良好的结果。此外,Campion等人(2021年a)提出了基于结构学习的树搜索方法。这得益于一种更广泛的变化信息传递方法,以积极推断(Champion等人,2021年b),使Bayesian网络的构成结构能够进行积极的推断。然而,这个信息传递树搜索方法,我们称之为分支-时间活跃推理(BTAI,20202020年a),在结构解析过程中提出了一种方法的实验性研究方法(Campion et al.),在结构解析剂方面,2021年a),这得益于一种更大的变化信息传递方法(Campionionionion 等) 方法,在这个背景下,我们展示了先前的精确度选择和Blientalalalalalalalalalalal,我们比较了一种B图中,让我们可以比较了一种Blial 。Blistal 。一个Blial