We propose a novel low-complexity lidar gesture recognition system for mobile robot control robust to gesture variation. Our system uses a modular approach, consisting of a pose estimation module and a gesture classifier. Pose estimates are predicted from lidar scans using a Convolutional Neural Network trained using an existing stereo-based pose estimation system. Gesture classification is accomplished using a Long Short-Term Memory network and uses a sequence of estimated body poses as input to predict a gesture. Breaking down the pipeline into two modules reduces the dimensionality of the input, which could be lidar scans, stereo imagery, or any other modality from which body keypoints can be extracted, making our system lightweight and suitable for mobile robot control with limited computing power. The use of lidar contributes to the robustness of the system, allowing it to operate in most outdoor conditions, to be independent of lighting conditions, and for input to be detected 360 degrees around the robot. The lidar-based pose estimator and gesture classifier use data augmentation and automated labeling techniques, requiring a minimal amount of data collection and avoiding the need for manual labeling. We report experimental results for each module of our system and demonstrate its effectiveness by testing it in a real-world robot teleoperation setting.
翻译:我们提出一个新的低复杂里拉手势识别系统,用于机动机器人控制,使其适应于手势变异。我们的系统使用模块化方法,包括一个表面估计模块和一个手势分类器。通过使用现有立体表面估计系统培训的进化神经网络,从利达尔扫描中预测Pose的估计数。通过长期短期内存网络完成定位分类,并使用一个估计体积序列作为预测动作的投入。将管道分成两个模块,可以减少输入的维度,这可以是利达尔扫描、立体图像或任何其他模式,从中提取身体关键点,使我们的系统较轻,适合以有限的计算能力进行移动机器人控制。使用利达尔有助于系统稳健,使其能够在大多数室外条件下运作,独立于照明条件,投入被检测到机器人周围360度。基于利达尔的姿势测测和姿态定分解器使用数据增强和自动标签技术,这可能需要最低限度的数据收集量,并避免人工标签的需要。我们报告每个系统的实际操作性测试模型的实验结果。我们报告每个系统在实际操作性测试过程中的试验结果。