To facilitate recent advances in robotics and AI for delicate collaboration between humans and machines, we propose the Kinova Gemini, an original robotic system that integrates conversational AI dialogue and visual reasoning to make the Kinova Gen3 lite robot help people retrieve objects or complete perception-based pick-and-place tasks. When a person walks up to Kinova Gen3 lite, our Kinova Gemini is able to fulfill the user's requests in three different applications: (1) It can start a natural dialogue with people to interact and assist humans to retrieve objects and hand them to the user one by one. (2) It detects diverse objects with YOLO v3 and recognize color attributes of the item to ask people if they want to grasp it via the dialogue or enable the user to choose which specific one is required. (3) It applies YOLO v3 to recognize multiple objects and let you choose two items for perception-based pick-and-place tasks such as "Put the banana into the bowl" with visual reasoning and conversational interaction.
翻译:为促进机器人和人工智能方面的最新进展,我们建议采用Kirova Gemini, 这是一种原始的机器人系统,将对话性AI对话与视觉推理结合起来,使Kirova Gen3 立体机器人帮助人们检索物体或完成基于感知的选址任务。 当一个人走到Kinova Gen3 立体时, 我们的基诺瓦 Gemini能够用三种不同的应用程序满足用户的要求:(1) 它可以开始与人进行自然对话,以便与人进行互动,协助人类逐个取回物体并将其交给用户。 (2) 它与YOLO v3 检测不同对象,并承认该项目的颜色属性,以便请人们通过对话来抓住它,或让用户能够选择需要的某个特定对象。 (3) 它应用YOLO v3 来识别多个对象,并允许你选择两个基于感知性的选位任务,例如用视觉推理和谈话性互动来“把香蕉放到碗里”等。