Traditional visual place recognition (VPR), usually using standard cameras, is easy to fail due to glare or high-speed motion. By contrast, event cameras have the advantages of low latency, high temporal resolution, and high dynamic range, which can deal with the above issues. Nevertheless, event cameras are prone to failure in weakly textured or motionless scenes, while standard cameras can still provide appearance information in this case. Thus, exploiting the complementarity of standard cameras and event cameras can effectively improve the performance of VPR algorithms. In the paper, we propose FE-Fusion-VPR, an attention-based multi-scale network architecture for VPR by fusing frames and events. First, the intensity frame and event volume are fed into the two-stream feature extraction network for shallow feature fusion. Next, the three-scale features are obtained through the multi-scale fusion network and aggregated into three sub-descriptors using the VLAD layer. Finally, the weight of each sub-descriptor is learned through the descriptor re-weighting network to obtain the final refined descriptor. Experimental results show that on the Brisbane-Event-VPR and DDD20 datasets, the Recall@1 of our FE-Fusion-VPR is 29.26% and 33.59% higher than Event-VPR and Ensemble-EventVPR, and is 7.00% and 14.15% higher than MultiRes-NetVLAD and NetVLAD. To our knowledge, this is the first end-to-end network that goes beyond the existing event-based and frame-based SOTA methods to fuse frame and events directly for VPR.
翻译:通常使用标准摄像头的传统视觉位置识别(VPR)通常很容易因光栅或高速运动而失败。相反,事件相机具有低延度、高时间分辨率和高动态范围的优势,可以处理上述问题。不过,事件相机在微弱的纹理或无运动的场景中容易失灵,而标准相机仍然可以在此情况下提供外观信息。因此,利用标准相机和事件相机的互补性可以有效地改进VPR算法的性能。在文件中,我们提议为VPR建立基于关注的多级网络结构,即使用框架和事件来超越VPR。首先,将强度框架和事件量输入两流特征提取网络,用于浅质聚合或无运动的场景。随后,通过多级集成网络获得三个规模的特征,并使用VLAD层,从而可以有效地改进 VPR-V的演算。最后的解调标码、 PR-20 和RV-RDF框架的重重重重重重度网络,这是我们当前DR-R-R-R-D-D-D% 和R-D-D-D-D-D-D-D-AD-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-I-D-I-I-I-I-I-I-I-I-I-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-O-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D-D