Visual odometry is a fundamental task for many applications on mobile devices and robotic platforms. Since such applications are oftentimes not limited to predefined target domains and learning-based vision systems are known to generalize poorly to unseen environments, methods for continual adaptation during inference time are of significant interest. In this work, we introduce CoVIO for online continual learning of visual-inertial odometry. CoVIO effectively adapts to new domains while mitigating catastrophic forgetting by exploiting experience replay. In particular, we propose a novel sampling strategy to maximize image diversity in a fixed-size replay buffer that targets the limited storage capacity of embedded devices. We further provide an asynchronous version that decouples the odometry estimation from the network weight update step enabling continuous inference in real time. We extensively evaluate CoVIO on various real-world datasets demonstrating that it successfully adapts to new domains while outperforming previous methods. The code of our work is publicly available at http://continual-slam.cs.uni-freiburg.de.
翻译:视觉里程计是移动设备和机器人平台上许多应用的基础任务。由于这些应用不限于预定义的目标领域,而且已知的基于学习的视觉系统容易在未看到的环境下泛化,因此在推理时进行连续适应的方法具有重要意义。在这项工作中,我们介绍了基于在线连续学习的视觉-惯性里程计(CoVIO)。CoVIO通过利用经验再现,在适应新域的同时,有效地减小了灾难性遗忘。特别是,我们提出了一种新的采样策略,以在固定大小的重播缓冲区中最大化图像多样性,以满足嵌入式设备的存储容量限制。我们进一步提供了一种异步版本,将里程计估计与网络权重更新步骤分离,实现了实时连续推理。我们在各种实际数据集上对CoVIO进行了广泛评估,证明了它成功地适应了新领域,并优于以前的方法。我们的工作代码公开可用,网址为http://continual-slam.cs.uni-freiburg.de。