Simultaneous Localization and Mapping (SLAM) system typically employ vision-based sensors to observe the surrounding environment. However, the performance of such systems highly depends on the ambient illumination conditions. In scenarios with adverse visibility or in the presence of airborne particulates (e.g. smoke, dust, etc.), alternative modalities such as those based on thermal imaging and inertial sensors are more promising. In this paper, we propose the first complete thermal-inertial SLAM system which combines neural abstraction in the SLAM front end with robust pose graph optimization in the SLAM back end. We model the sensor abstraction in the front end by employing probabilistic deep learning parameterized by Mixture Density Networks (MDN). Our key strategies to successfully model this encoding from thermal imagery are the usage of normalized 14-bit radiometric data, the incorporation of hallucinated visual (RGB) features, and the inclusion of feature selection to estimate the MDN parameters. To enable a full SLAM system, we also design an efficient global image descriptor which is able to detect loop closures from thermal embedding vectors. We performed extensive experiments and analysis using three datasets, namely self-collected ground robot and handheld data taken in indoor environment, and one public dataset (SubT-tunnel) collected in underground tunnel. Finally, we demonstrate that an accurate thermal-inertial SLAM system can be realized in conditions of both benign and adverse visibility.
翻译:同步本地化和绘图系统(SLAM)通常使用基于视觉的传感器来观察周围环境,但是,这些系统的性能高度取决于周围的照明条件;在可见度不利或空气中的颗粒(如烟、尘等)出现时,以热成像和惯性传感器为基础的替代方法比较有希望;在本文件中,我们提议了第一个完整的热-内皮SLAM系统,该系统将SLAM前端的神经抽取与SLAM后端的强力成形图优化结合起来。我们用混合密度网络(MDN)的概率性深学习参数来模拟前端的传感器抽取。我们成功模拟热成像的编码的关键战略是使用常规14位辐射测量数据和惯性视觉(RGB)特性,以及纳入地貌选择来估计MDN参数。为了能够建立一个完整的SLMMM系统,我们还设计了一个高效的全球图层图解仪,它能够从热嵌入的自我闭路槽中探测到热嵌入的精确闭路的闭路路径。我们进行了广泛的实验,在一次实地实验室和实地环境中进行数据分析。