Recently, several works achieve end-to-end visual servoing (VS) for robotic manipulation by replacing traditional controller with differentiable neural networks, but lose the ability to servo arbitrary desired poses. This letter proposes a differentiable architecture for arbitrary pose servoing: a hyper-network based neural controller (HPN-NC). To achieve this, HPN-NC consists of a hyper net and a low-level controller, where the hyper net learns to generate the parameters of the low-level controller and the controller uses the 2D keypoints error for control like traditional image-based visual servoing (IBVS). HPN-NC can complete 6 degree of freedom visual servoing with large initial offset. Taking advantage of the fully differentiable nature of HPN-NC, we provide a three-stage training procedure to servo real world objects. With self-supervised end-to-end training, the performance of the integrated model can be further improved in unseen scenes and the amount of manual annotations can be significantly reduced.
翻译:最近,一些工作通过将传统控制器替换为可微分神经网络实现了机器人操作的端到端视觉伺服,但失去了伺服任意姿态的能力。本文提出了一种用于任意姿态伺服的可微架构:基于超网络的神经控制器(HPN-NC)。为了实现这一目标,HPN-NC包括一个超网络和一个低层控制器,其中超网络学习生成低层控制器的参数,而控制器使用2D关键点误差进行控制,就像传统的基于图像的视觉伺服(IBVS)一样。 HPN-NC可以完成在初始偏移较大的情况下的六个自由度的视觉伺服。利用HPN-NC完全可微的特性,我们提供了一个三阶段的训练过程来伺服现实世界的物体。通过自监督的端到端训练,综合模型的性能可以进一步在未见过的场景中得到改善,并且手动标注的数量可以显著降低。