RxR-Habit 愿景和语言导航竞争第一地点解决办法(CVPR 2022) (1st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022))

This report presents the methods of the winning entry of the RxR-Habitat Competition in CVPR 2022. The competition addresses the problem of Vision-and-Language Navigation in Continuous Environments (VLN-CE), which requires an agent to follow step-by-step natural language instructions to reach a target. We present a modular plan-and-control approach for the task. Our model consists of three modules: the candidate waypoints predictor (CWP), the history enhanced planner and the tryout controller. In each decision loop, CWP first predicts a set of candidate waypoints based on depth observations from multiple views. It can reduce the complexity of the action space and facilitate planning. Then, a history-enhanced planner is adopted to select one of the candidate waypoints as the subgoal. The planner additionally encodes historical memory to track the navigation progress, which is especially effective for long-horizon navigation. Finally, we propose a non-parametric heuristic controller named tryout to execute low-level actions to reach the planned subgoal. It is based on the trial-and-error mechanism which can help the agent to avoid obstacles and escape from getting stuck. All three modules work hierarchically until the agent stops. We further take several recent advances of Vision-and-Language Navigation (VLN) to improve the performance such as pretraining based on large-scale synthetic in-domain dataset, environment-level data augmentation and snapshot model ensemble. Our model won the RxR-Habitat Competition 2022, with 48% and 90% relative improvements over existing methods on NDTW and SR metrics respectively.

翻译：本报告介绍了在CVPR 2022年CVPR 中RxR-Higend竞争获胜的入场方法。竞争涉及连续环境中的视觉和语言导航(VLN-CE)问题,这一问题要求代理机构逐步遵循自然语言指令,以达到目标。我们为任务提出了一个模块式的计划和控制方法。我们的模型由三个模块组成:候选路点预测(CWP)、历史强化计划仪和试运行控制器。在每个决策循环中,CWP首先根据多重观点的深度观测预测一套候选路径点。它可以降低行动空间的复杂性,并促进规划。然后,采用历史强化计划家来选择一个候选路径点,作为次级目标。我们为跟踪导航进度,这对长程导航导航特别有效。最后,我们建议一个非参数性能超常的超常控制器,根据测试的48度内位标准标准运行,从最近测试的Siral-rliver数据级升级升级到前的Rioral-ral-ldalligal-ral-lgal-moal-ligal-ligal-modeal-ligal-ligal-ligal la-ligal-liver-ligal-leval-L-leval-L-leval-leval-lation-l-l-l-lational-lg-leval-l-l-l-lgal-l-l-l-l-l-lx-lation-lx-lx-l-lgal-lg-lation-lation-lation-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-