LiDAR has become one of the primary sensors in robotics and autonomous system for high-accuracy situational awareness. In recent years, multi-modal LiDAR systems emerged, and among them, LiDAR-as-a-camera sensors provide not only 3D point clouds but also fixed-resolution 360{\deg}panoramic images by encoding either depth, reflectivity, or near-infrared light in the image pixels. This potentially brings computer vision capabilities on top of the potential of LiDAR itself. In this paper, we are specifically interested in utilizing LiDARs and LiDAR-generated images for tracking Unmanned Aerial Vehicles (UAVs) in real-time which can benefit applications including docking, remote identification, or counter-UAV systems, among others. This is, to the best of our knowledge, the first work that explores the possibility of fusing the images and point cloud generated by a single LiDAR sensor to track a UAV without a priori known initialized position. We trained a custom YOLOv5 model for detecting UAVs based on the panoramic images collected in an indoor experiment arena with a MOCAP system. By integrating with the point cloud, we are able to continuously provide the position of the UAV. Our experiment demonstrated the effectiveness of the proposed UAV tracking approach compared with methods based only on point clouds or images. Additionally, we evaluated the real-time performance of our approach on the Nvidia Jetson Nano, a popular mobile computing platform.
翻译:Lidar已成为机器人和自主系统中用于高精度情境感知的主要传感器之一。近年来,多模态激光雷达系统逐渐出现,其中,作为相机的LiDAR传感器不仅提供3D点云,还可以通过在图像像素中编码深度、反射性或近红外光来提供固定分辨率的360度全景图像。这可能使LiDAR本身具备计算机视觉能力。在本文中,我们专门探讨了在实时中利用LiDAR和LiDAR生成的图像来跟踪无人机(UAV)的可能性,这可以使应用程序受益,包括对接、远程识别或反UAV系统,等等。就我们所知,这是第一个探索将由单个LiDAR传感器生成的图像和点云融合以跟踪无人机的工作,而且不需要先验已知位姿。我们训练了一个定制的YOLOv5模型,以根据MOCAP系统收集的室内实验场地的全景图像检测无人机。通过与点云集成,我们能够不断提供无人机的位置。我们的实验证明了所提出的无人机跟踪方法的有效性,与仅基于点云或图像的方法相比。此外,我们还在Nvidia Jetson Nano上评估了我们的方法的实时性能,该平台是一种常用的移动计算平台。