Benchmarking Simultaneous Localization and Mapping (SLAM) algorithms is important to scientists and users of robotic systems alike. But through their many configuration options in hardware and software, SLAM systems feature a vast parameter space that scientists up to now were not able to explore. The proposed SLAM Hive Benchmarking Suite is able to analyze SLAM algorithms in 1000's of mapping runs, through its utilization of container technology and deployment in a cluster. This paper presents the architecture and open source implementation of SLAM Hive and compares it to existing efforts on SLAM evaluation. Furthermore, we highlight the function of SLAM Hive by exploring some open source algorithms on public datasets in terms of accuracy. We compare the algorithms against each other and evaluate how parameters effect not only accuracy but also CPU and memory usage. Through this we show that SLAM Hive can become an essential tool for proper comparisons and evaluations of SLAM algorithms and thus drive the scientific development in the research on SLAM.
翻译:通过硬件和软件的多种配置选项,SLAM系统具有广阔的参数空间,因此对SLAM算法进行基准测试具有重要意义。但是迄今为止,科学家们不能够探索SLAM系统的整个参数空间。本文介绍了SLAM Hive的体系结构和开源实现,并将其与现有SLAM评估工具进行了比较。此外,我们通过研究一些公共数据集上的开放源代码算法的准确性,重点介绍了SLAM Hive的功能。我们相互比较算法,并评估参数对准确性以及CPU和内存使用率的影响。通过这一研究,我们表明SLAM Hive可成为适当比较和评估SLAM算法,在SLAM研究领域推动科学发展的重要工具。