The co-adaptation of robot morphology and behaviour becomes increasingly important with the advent of fast 3D-manufacturing methods and efficient deep reinforcement learning algorithms. A major challenge for the application of co-adaptation methods to the real world is the simulation-to-reality-gap due to model and simulation inaccuracies. However, prior work focuses primarily on the study of evolutionary adaptation of morphologies exploiting analytical models and (differentiable) simulators with large population sizes, neglecting the existence of the simulation-to-reality-gap and the cost of manufacturing cycles in the real world. This paper presents a new approach combining classic high-frequency deep neural networks with computational expensive Graph Neural Networks for the data-efficient co-adaptation of agents with varying numbers of degrees-of-freedom. Evaluations in simulation show that the new method can co-adapt agents within such a limited number of production cycles by efficiently combining design optimization with offline reinforcement learning, that it allows for the direct application to real-world co-adaptation tasks in future work
翻译:随着快速的3D制造方法的出现和高效的深度强化学习算法的出现,机器人形态学和行为的共同适应变得日益重要。将共同适应方法应用到现实世界的一个主要挑战是模型和模拟不准确造成的模拟到现实差距。然而,先前的工作主要侧重于研究利用分析模型和人口规模庞大的(可区别的)模拟模型和(可区别的)模拟器对形态进行进化适应,忽视模拟到真实世界中存在的模拟到现实差距和制造周期的成本。本文介绍了一种新方法,将典型的高频深度神经网络与计算昂贵的图象神经网络结合起来,用于不同程度自由的制剂的数据高效共同适应。模拟评价显示,新方法可以通过高效地将设计优化与离线强化学习结合起来,在如此有限的生产周期中使新的方法能够对设计优化和离线强化学习进行共适应,从而能够直接应用于未来工作中的现实世界共同适应任务。