Simultaneous localization and mapping (SLAM) are crucial for autonomous robots (e.g., self-driving cars, autonomous drones), 3D mapping systems, and AR/VR applications. This work proposed a novel LiDAR-inertial-visual fusion framework termed R$^3$LIVE++ to achieve robust and accurate state estimation while simultaneously reconstructing the radiance map on the fly. R$^3$LIVE++ consists of a LiDAR-inertial odometry (LIO) and a visual-inertial odometry (VIO), both running in real-time. The LIO subsystem utilizes the measurements from a LiDAR for reconstructing the geometric structure (i.e., the positions of 3D points), while the VIO subsystem simultaneously recovers the radiance information of the geometric structure from the input images. R$^3$LIVE++ is developed based on R$^3$LIVE and further improves the accuracy in localization and mapping by accounting for the camera photometric calibration (e.g., non-linear response function and lens vignetting) and the online estimation of camera exposure time. We conduct more extensive experiments on both public and our private datasets to compare our proposed system against other state-of-the-art SLAM systems. Quantitative and qualitative results show that our proposed system has significant improvements over others in both accuracy and robustness. In addition, to demonstrate the extendability of our work, {we developed several applications based on our reconstructed radiance maps, such as high dynamic range (HDR) imaging, virtual environment exploration, and 3D video gaming.} Lastly, to share our findings and make contributions to the community, we make our codes, hardware design, and dataset publicly available on our Github: github.com/hku-mars/r3live
翻译:同步本地化和映射( SLAM) 对自主机器人( 如自驾驶汽车、自驾驶无人机)、 3D 虚拟绘图系统、 AR/ VR 应用程序至关重要。 这项工作提出了名为 R$3$LIVE+的新颖的LIDAR- 内脏- 视觉融合框架, 以在同时重建苍蝇上的亮度地图的同时实现稳健和准确的状态估算。 R$3$LIVE+( LIO) 包括一个LiDAR- 内脏odology( LIO) 和一个视觉- 内脏测量( VIO) 。 LIO 子系统利用LIDAR 的测量来重建地理结构( 即 3D 点的位置), 而VIO 系统同时从输入图像中恢复地球光度结构的亮度信息 。 R$3$LIVE+( LIO) 以Rualtime( 3$LIO) 和图像校验的精确度和图像校验校准( e- sal- ladeal laveal laftal laftal laisal laisal) ladeal ladeal) ex), 和图像( laveal lading lax- sal laveal d) laveal- sal- sal- sal) laveal dal laveal dal dal dal dismal disal disal disal), lading ( lading lading) labal labal disal disal disal disal disal des) labal dess) des) des) des),,, labal labal labal labal labal labal lab lab lab) lab 和我们制, lab labal labal, labal- sal lad lad lab labal lab labal labal 和我们的多项, 和我们用系统, labal