Transparent objects are widely used in our daily lives and therefore robots need to be able to handle them. However, transparent objects suffer from light reflection and refraction, which makes it challenging to obtain the accurate depth maps required to perform handling tasks. In this paper, we propose a novel affordance-based framework for depth reconstruction and manipulation of transparent objects, named A4T. A hierarchical AffordanceNet is first used to detect the transparent objects and their associated affordances that encode the relative positions of an object's different parts. Then, given the predicted affordance map, a multi-step depth reconstruction method is used to progressively reconstruct the depth maps of transparent objects. Finally, the reconstructed depth maps are employed for the affordance-based manipulation of transparent objects. To evaluate our proposed method, we construct a real-world dataset TRANS-AFF with affordances and depth maps of transparent objects, which is the first of its kind. Extensive experiments show that our proposed methods can predict accurate affordance maps, and significantly improve the depth reconstruction of transparent objects compared to the state-of-the-art method, with the Root Mean Squared Error in meters significantly decreased from 0.097 to 0.042. Furthermore, we demonstrate the effectiveness of our proposed method with a series of robotic manipulation experiments on transparent objects. See supplementary video and results at https://sites.google.com/view/affordance4trans.
翻译:透明天体在我们日常生活中被广泛使用,因此机器人需要能够处理这些天体。但是,透明天体受到光线反射和折射的影响,因此要获得执行任务所需的准确深度地图就具有挑战性。在本文中,我们提议为深度重建和操作透明天体提出一个新的基于价格的框架,称为A4TT。一个等级的AffordanceNet首先用来探测透明天体及其相关天体,以编码一个天体不同部分的相对位置。然后,根据预测的可负担性地图,采用多步深深度重建方法逐步重建透明天体的深度地图。最后,利用重建的深度地图来进行基于价格的透明天体操作。为了评估我们拟议的方法,我们用价格和深度图绘制透明天体的真-世界数据集TRANS-FF,这是它的第一个类型。广泛的实验表明,我们提出的方法可以预测准确的支付天体图,并大大改进透明天体的深度重建,比着透明天体的深度地图。最后,用根基的平方位4 4 4 地图显示我们从40米的原始天体错误序列的透明性实验结果。