In this paper a deep learning architecture is presented that can, in real time, detect the 2D locations of certain landmarks of physical tools, such as a hammer or screwdriver. To avoid the labor of manual labeling, the network is trained on synthetically generated data. Training computer vision models on computer generated images, while still achieving good accuracy on real images, is a challenge due to the difference in domain. The proposed method uses an advanced rendering method in combination with transfer learning and an intermediate supervision architecture to address this problem. It is shown that the model presented in this paper, named Intermediate Heatmap Model (IHM), generalizes to real images when trained on synthetic data. To avoid the need for an exact textured 3D model of the tool in question, it is shown that the model will generalize to an unseen tool when trained on a set of different 3D models of the same type of tool. IHM is compared to two existing approaches to keypoint detection and it is shown that it outperforms those at detecting tool landmarks, trained on synthetic data.
翻译:本文介绍了一个深层次的学习结构,它可以实时探测诸如锤子或螺丝起子等物理工具的某些里程碑的2D位置。为了避免人工标签的劳动,对网络进行了合成生成数据的培训。培训计算机生成图像的计算机视觉模型,虽然在真实图像上仍然实现良好的准确性,但由于领域差异,这是一个挑战。拟议方法使用先进的制作方法,结合转移学习和中间监督结构来解决这一问题。显示本文中介绍的模式,名为中间热马普模型(HM),在进行合成数据培训时,一般化为真实图像。为避免需要该工具的精确的3D模型,显示该模型在就同一类型工具的一套不同的3D模型进行培训时,将概括为一种看不见的工具。将IHM与现有的两种关键点探测方法进行比较,并显示它比照了在检测工具标志上所接受的、经过合成数据培训的模型。