用于视觉导航的可移动神经辐射映射图</s> (Renderable Neural Radiance Map for Visual Navigation)

from arxiv, Preprint version, CVPR 2023 accepted. Supplementary Video: https://youtu.be/DHlcKbVDt5A. This will be replaced by a camera-ready version with some minor revisions

We propose a novel type of map for visual navigation, a renderable neural radiance map (RNR-Map), which is designed to contain the overall visual information of a 3D environment. The RNR-Map has a grid form and consists of latent codes at each pixel. These latent codes are embedded from image observations, and can be converted to the neural radiance field which enables image rendering given a camera pose. The recorded latent codes implicitly contain visual information about the environment, which makes the RNR-Map visually descriptive. This visual information in RNR-Map can be a useful guideline for visual localization and navigation. We develop localization and navigation frameworks that can effectively utilize the RNR-Map. We evaluate the proposed frameworks on camera tracking, visual localization, and image-goal navigation. Experimental results show that the RNR-Map-based localization framework can find the target location based on a single query image with fast speed and competitive accuracy compared to other baselines. Also, this localization framework is robust to environmental changes, and even finds the most visually similar places when a query image from a different environment is given. The proposed navigation framework outperforms the existing image-goal navigation methods in difficult scenarios, under odometry and actuation noises. The navigation framework shows 65.7% success rate in curved scenarios of the NRNS dataset, which is an improvement of 18.6% over the current state-of-the-art.

翻译：我们提出新型视觉导航地图,即可变神经光亮地图(RNR-Map),用于包含3D环境的总体视觉信息。RNR-Map具有网格形式,由每个像素的潜伏代码组成。这些潜伏代码从图像观测中嵌入,并可以转换为神经光亮场,使图像能够形成相机。所记录的潜伏代码隐含着关于环境的视觉信息,使RNR-Map能够进行视觉描述。RNR-Map中的这种视觉信息可以成为视觉定位和导航的有用指南。我们开发了可有效利用RNR-Map的本地化和导航框架。我们评价了关于相机跟踪、视觉本地化和图像目标导航的拟议框架。实验结果表明,RNR-Map本地化框架可以找到基于单一查询图像的目标位置,其速度和竞争准确性与其他基线相比。此外,RNR-M-M-M-Map的本地化框架对于环境的视觉变化十分有力,而且当从不同的环境查询图像中找到最视觉相似的地方。我们开发RNR-M-M-M-Map-M-M-M-M-M-M-M-M-M-M-M-M-M-M-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-S-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L-L</s>