Enabling additive manufacturing to employ a wide range of novel, functional materials can be a major boost to this technology. However, making such materials printable requires painstaking trial-and-error by an expert operator, as they typically tend to exhibit peculiar rheological or hysteresis properties. Even in the case of successfully finding the process parameters, there is no guarantee of print-to-print consistency due to material differences between batches. These challenges make closed-loop feedback an attractive option where the process parameters are adjusted on-the-fly. There are several challenges for designing an efficient controller: the deposition parameters are complex and highly coupled, artifacts occur after long time horizons, simulating the deposition is computationally costly, and learning on hardware is intractable. In this work, we demonstrate the feasibility of learning a closed-loop control policy for additive manufacturing using reinforcement learning. We show that approximate, but efficient, numerical simulation is sufficient as long as it allows learning the behavioral patterns of deposition that translate to real-world experiences. In combination with reinforcement learning, our model can be used to discover control policies that outperform baseline controllers. Furthermore, the recovered policies have a minimal sim-to-real gap. We showcase this by applying our control policy in-vivo on a single-layer, direct ink writing printer.
翻译:使添加剂制造业能够采用多种新型功能性材料,可以大大推动这一技术。然而,使此类材料能够印刷,需要专家操作员进行艰苦的试探,因为这些材料通常会表现出特殊的风湿或歇斯底里特性。即使成功地找到工艺参数,由于批量之间的物质差异,也无法保证印刷到印刷的一致性。这些挑战使闭路反馈成为一种具有吸引力的选择,因为流程参数在飞行时调整。设计高效控制器面临若干挑战:沉积参数复杂且高度结合,文物在很长的时间跨度之后出现,模拟沉积成本很高,硬件学习也很棘手。在这项工作中,我们展示了学习闭路控制政策的可行性,以便利用强化学习添加剂制造。我们表明,只要能够学习转换成现实世界经验的沉积行为模式,近似但有效的数字模拟就足够了。与强化学习相结合,我们的模型可以用来发现超越基准控制器的缺口政策。此外,我们所回收的软件政策在直接的平面上展示了我们的一种标准。