Human-robot co-manipulation of soft materials, such as fabrics, composites, and sheets of paper/cardboard, is a challenging operation that presents several relevant industrial applications. Estimating the deformation state of the co-manipulated material is one of the main challenges. Viable methods provide the indirect measure by calculating the human-robot relative distance. In this paper, we develop a data-driven model to estimate the deformation state of the material from a depth image through a Convolutional Neural Network (CNN). First, we define the deformation state of the material as the relative roto-translation from the current robot pose and a human grasping position. The model estimates the current deformation state through a Convolutional Neural Network, specifically a DenseNet-121 pretrained on ImageNet.The delta between the current and the desired deformation state is fed to the robot controller that outputs twist commands. The paper describes the developed approach to acquire, preprocess the dataset and train the model. The model is compared with the current state-of-the-art method based on a skeletal tracker from cameras. Results show that our approach achieves better performances and avoids the various drawbacks caused by using a skeletal tracker.Finally, we also studied the model performance according to different architectures and dataset dimensions to minimize the time required for dataset acquisition
翻译:人-机器人软材料共同操作是一种具有多个重要工业应用的挑战性操作。估计共同操作材料的变形状态是主要难题之一。可行的方法是通过计算人-机器人相对距离提供间接测量。在本文中,我们通过卷积神经网络(CNN)开发了一种数据驱动模型,通过深度图像估计材料的变形状态。首先,我们将材料的变形状态定义为当前机器人姿势和人类抓取位置的相对旋转-平移关系。该模型通过卷积神经网络(特别是在ImageNet上预训练的DenseNet-121)估计当前的变形状态。当前与期望的变形状态之间的差异被馈送到机器人控制器,输出扭转命令。本文介绍了开发方法以获得、预处理数据集并训练模型。该模型与基于摄像机骨骼跟踪器的当前最先进方法进行了比较。结果表明,我们的方法实现了更好的性能,并避免了因使用骨骼跟踪器所造成的各种缺点。最后,我们还研究了模型性能,根据不同的架构和数据集尺寸来最小化数据集获取所需的时间。