Neural cellular automata are a recent model used to model biological phenomena emerging from multicellular organisms. In these models, artificial neural networks are used as update rules for cellular automata. They are end-to-end differentiable systems where the parameters of the neural network can be optimized to achieve a particular task. In this work, we used them to control a cart-pole agent. The observations of the environment are transmitted in inputs cells, while the values of output cells are used as readout of the system. We trained the model using deep-Q learning, where the states of the output cells were used as the q-value estimates to be optimized. We found that the computing abilities of the cellular automata were maintained over several hundreds of thousands of iterations, producing an emergent stable behavior in the environment it controls for thousands of steps. Moreover, the system demonstrated life-like phenomena such as a developmental phase, regeneration after damage, stability despite a noisy environment, and robustness to unseen disruption such as input deletion.
翻译:神经细胞自动数据是用来模拟多细胞有机体产生的生物现象的一种最新模型。 在这些模型中, 人工神经网络被用作细胞自动数据更新规则。 它们是一种端到端的不同系统, 可以优化神经网络的参数以完成特定任务。 在这项工作中, 我们用它们来控制一个推车柱剂。 对环境的观测是在输入细胞中传输的, 而输出细胞的值被用作系统的读取功能。 我们用深Q学习来培训模型, 其中输出细胞的状态被用作要优化的q值估计值。 我们发现, 细胞自动数据计算机的计算能力维持在数以万计的迭代值上, 在它控制的环境下, 产生出一个几千个步骤的突发稳定的行为。 此外, 系统展示了像生命一样的现象, 如发展阶段、 破坏后的再生、 破坏后的稳定性, 尽管环境很吵闹, 以及 强力去见不到的干扰, 比如删除输入。