Neural Radiance Fields (NeRFs) have recently emerged as a powerful paradigm for the representation of natural, complex 3D scenes. NeRFs represent continuous volumetric density and RGB values in a neural network, and generate photo-realistic images from unseen camera viewpoints through ray tracing. We propose an algorithm for navigating a robot through a 3D environment represented as a NeRF using only an on-board RGB camera for localization. We assume the NeRF for the scene has been pre-trained offline, and the robot's objective is to navigate through unoccupied space in the NeRF to reach a goal pose. We introduce a trajectory optimization algorithm that avoids collisions with high-density regions in the NeRF based on a discrete time version of differential flatness that is amenable to constraining the robot's full pose and control inputs. We also introduce an optimization based filtering method to estimate 6DoF pose and velocities for the robot in the NeRF given only an onboard RGB camera. We combine the trajectory planner with the pose filter in an online replanning loop to give a vision-based robot navigation pipeline. We present simulation results with a quadrotor robot navigating through a jungle gym environment, the inside of a church, and Stonehenge using only an RGB camera. We also demonstrate an omnidirectional ground robot navigating through the church, requiring it to reorient to fit through the narrow gap. Videos of this work can be found at https://mikh3x4.github.io/nerf-navigation/ .
翻译:最近,Neoral Reradiance Fields(Neoral Redian Fields)成为代表自然、复杂的三维场景的强大范例。 NeRFs代表神经网络中连续的体积密度和RGB值,并通过射线追踪从看不见的摄像器角度生成光现实图像。我们建议了一种算法,在3D环境中导航机器人,这个3D环境代表NERF, 仅使用机上 RGB 相机进行定位。我们假设现场的NERF是预先训练的离线,而机器人的视频的目标是通过NERF4 的无人空间导航,以达到一个目标。我们引入轨迹色优化算法,避免与NERF3 的高密度区域发生碰撞。我们引入了轨迹优化算法,该算法基于离散的时间版本,可以限制机器人的完整配置和控制投入。我们还引入了基于 NERF 6 DoF 的过滤法配置和机器人在 NERF 仅安装了RGB rodual RGB rodual rouptal roduction room rouptal 。我们找到了轨图图和图像过滤器, 。我们通过一个基于建筑的游戏的校内浏览的校内浏览的校内导航的模拟,我们通过模拟展示的游戏展示了一台的校造纸路,我们模拟的校造纸路路路路的模拟的模拟,我们的模拟展示结果。我们现在的校。