We propose a novel concept of augmented reality (AR) human-drone interaction driven by RL-based swarm behavior to achieve intuitive and immersive control of a swarm formation of unmanned aerial vehicles. The DroneARchery system developed by us allows the user to quickly deploy a swarm of drones, generating flight paths simulating archery. The haptic interface LinkGlide delivers a tactile stimulus of the bowstring tension to the forearm to increase the precision of aiming. The swarm of released drones dynamically avoids collisions between each other, the drone following the user, and external obstacles with behavior control based on deep reinforcement learning. The developed concept was tested in the scenario with a human, where the user shoots from a virtual bow with a real drone to hit the target. The human operator observes the ballistic trajectory of the drone in an AR and achieves a realistic and highly recognizable experience of the bowstring tension through the haptic display. The experimental results revealed that the system improves trajectory prediction accuracy by 63.3% through applying AR technology and conveying haptic feedback of pulling force. DroneARchery users highlighted the naturalness (4.3 out of 5 point Likert scale) and increased confidence (4.7 out of 5) when controlling the drone. We have designed the tactile patterns to present four sliding distances (tension) and three applied force levels (stiffness) of the haptic display. Users demonstrated the ability to distinguish tactile patterns produced by the haptic display representing varying bowstring tension(average recognition rate is of 72.8%) and stiffness (average recognition rate is of 94.2%). The novelty of the research is the development of an AR-based approach for drone control that does not require special skills and training from the operator.
翻译:我们提出了一个由RL驱动的群温行为驱动的扩大现实(AR)人类-地铁互动的新概念。 我们开发的DroneARchery系统使用户能够迅速部署一群无人机,产生飞行路径模拟箭头。 LinkGlide 的突触界面向前方显示弓纹紧张的触动性刺激,以提高目标的精确度。释放的无人机动态地避免了彼此碰撞、用户之后的无人机以及基于深度强化学习的行为控制外部障碍。我们开发的DroneARchery系统使用户能够迅速部署成群的无人机群群群,产生飞行路径模拟箭头。 LinkGlide 的突触性界面向前方显示弓纹紧张的触动性刺激力,以便提高目标的精确性。(释放的无人机群群群群群群群群群)动态地避免了相互碰撞、用户的无人机体碰撞,以及基于深度强化的动作控制模式的外部能力。 运行者需要从虚拟弓箭头射击的弹道轨道轨迹轨迹,通过随机的直径直径直径直径直径直径直径直径直径直径直径直径直径直径直径直径直地显示。