In this paper, we present a grammar-based natural language framework for robot programming, specifically for pick-and-place tasks. Our approach uses a custom dictionary of action words, designed to store together words that share meaning, allowing for easy expansion of the vocabulary by adding more action words from a lexical database. We validate our Natural Language Robot Programming (NLRP) framework through simulation and real-world experimentation, using a Franka Panda robotic arm equipped with a calibrated camera-in-hand and a microphone. Participants were asked to complete a pick-and-place task using verbal commands, which were converted into text using Google's Speech-to-Text API and processed through the NLRP framework to obtain joint space trajectories for the robot. Our results indicate that our approach has a high system usability score. The framework's dictionary can be easily extended without relying on transfer learning or large data sets. In the future, we plan to compare the presented framework with different approaches of human-assisted pick-and-place tasks via a comprehensive user study.
翻译:在本文中,我们提出了一种基于语法的自然语言框架,用于机器人编程,特别是用于拾取和放置任务。我们的方法使用自定义的动作词词典,旨在将共享意义的单词存储在一起,从而可以通过从词汇数据库中添加更多动作词轻松扩展词汇表。我们通过模拟和实际实验验证了我们的自然语言机器人编程(NLRP)框架,使用配备校准相机式夹持装置和麦克风的Franka Panda机器人臂。参与者被要求使用口头命令完成拾取和放置任务,这些命令使用Google的语音转文本API转换为文本,并通过NLRP框架进行处理,以获取机器人的关节空间轨迹。我们的结果表明,我们的方法具有较高的系统可用性评分。该框架的词典可以轻松扩展,而无需依赖于迁移学习或大型数据集。在未来,我们计划通过全面的用户研究比较所提出的框架与不同的人类辅助拾取和放置任务的方法。