Pneumatic soft robots present many advantages in manipulation tasks. Notably, their inherent compliance makes them safe and reliable in unstructured and fragile environments. However, full-body shape sensing for pneumatic soft robots is challenging because of their high degrees of freedom and complex deformation behaviors. Vision-based proprioception sensing methods relying on embedded cameras and deep learning provide a good solution to proprioception sensing by extracting the full-body shape information from the high-dimensional sensing data. But the current training data collection process makes it difficult for many applications. To address this challenge, we propose and demonstrate a robust sim-to-real pipeline that allows the collection of the soft robot's shape information in high-fidelity point cloud representation. The model trained on simulated data was evaluated with real internal camera images. The results show that the model performed with averaged Chamfer distance of 8.85 mm and tip position error of 10.12 mm even with external perturbation for a pneumatic soft robot with a length of 100.0 mm. We also demonstrated the sim-to-real pipeline's potential for exploring different configurations of visual patterns to improve vision-based reconstruction results. The code and dataset are available at https://github.com/DeepSoRo/DeepSoRoSim2Real.
翻译:气动软机器人在操作任务中有许多优势。 值得注意的是,它们的内在合规性使其在结构不健全和脆弱的环境中变得安全可靠。 然而,对气动软机器人的全体形状感测因其自由程度高和复杂的变形行为而具有挑战性。 依靠嵌入相机和深层学习的基于视觉的自我感知感知感测方法为自我感测提供了一个很好的解决方案,从高维感测数据中提取全体形状信息。 但当前的培训数据收集过程使许多应用程序都难以应对这一挑战。 为了应对这一挑战,我们提议并展示一个强大的模拟到真实的管道,以便能够在高纤维点云中收集软机器人的形状信息。对模拟数据培训的模型用真正的内部相机图像进行了评估。结果显示,该模型以8.85毫米的平均开关距离和10.12毫米的底位差差差差错误进行演演演,即使对100.0毫米长的充气软体机器人进行外部扰动。 我们还展示了探索不同视觉模式的Simto-真实管道潜力。</s>