There have been many studies in robotics to improve the communication skills of domestic service robots. Most studies, however, have not fully benefited from recent advances in deep neural networks because the training datasets are not large enough. In this paper, our aim is to augment the datasets based on a crossmodal language generation model. We propose the Case Relation Transformer (CRT), which generates a fetching instruction sentence from an image, such as "Move the blue flip-flop to the lower left box." Unlike existing methods, the CRT uses the Transformer to integrate the visual features and geometry features of objects in the image. The CRT can handle the objects because of the Case Relation Block. We conducted comparison experiments and a human evaluation. The experimental results show the CRT outperforms baseline methods.
翻译:对机器人进行了许多研究,以提高家庭服务机器人的沟通技能。然而,大多数研究并没有从深神经网络最近的进步中充分获益,因为培训数据集不够大。在本文中,我们的目标是增加基于跨模式语言生成模型的数据集。我们建议采用Case Relation 变换器(CRT),从图像中提取指令句,如“将蓝色翻转式移动到左下方框 ” 。与现有方法不同, CRT使用变换器整合图像中对象的视觉特征和几何特征。CRT可以使用CR 处理对象,因为CRelation Block。我们进行了比较实验和人类评估。实验结果显示CRT优于基线方法。