Reliability of SLAM systems is considered one of the critical requirements in modern autonomous systems. This directed the efforts to developing many state-of-the-art systems, creating challenging datasets, and introducing rigorous metrics to measure SLAM performance. However, the link between datasets and performance in the robustness/resilience context has rarely been explored. In order to fill this void, characterization of the operating conditions of SLAM systems is essential in order to provide an environment for quantitative measurement of robustness and resilience. In this paper, we argue that for proper evaluation of SLAM performance, the characterization of SLAM datasets serves as a critical first step. The study starts by reviewing previous efforts for quantitative characterization of SLAM datasets. Then, the problem of perturbation characterization is discussed and the linkage to SLAM robustness/resilience is established. After that, we propose a novel, generic and extendable framework for quantitative analysis and comparison of SLAM datasets. Additionally, a description of different characterization parameters is provided. Finally, we demonstrate the application of our framework by presenting the characterization results of three SLAM datasets: KITTI, EuroC-MAV, and TUM-VI highlighting the level of insights achieved by the proposed framework.
翻译:为了填补这一空白,对SLM系统的运行条件进行定性至关重要,以便为对SLM的稳健性和复原力进行定量衡量提供一种环境;在本文件中,我们主张,为了对SLM的绩效进行适当评估,首先必须确定SLM数据集的定性,作为关键的第一步。这项研究首先审查了以前对SLM数据集进行定量定性的工作,然后讨论了扰动性定性的问题,确定了与SLM系统稳健性/复原力的联系。之后,我们提出了一个新的、通用的和可扩展的框架,用于对SLM数据集进行定量分析和比较。此外,我们介绍了不同的定性参数。最后,我们通过提出三个SLMMMT的定性结果,展示了我们框架的应用情况。