Rejecting correspondence outliers enables to boost the correspondence quality, which is a critical step in achieving high point cloud registration accuracy. The current state-of-the-art correspondence outlier rejection methods only utilize the structure features of the correspondences. However, texture information is critical to reject the correspondence outliers in our human vision system. In this paper, we propose General Multimodal Fusion (GMF) to learn to reject the correspondence outliers by leveraging both the structure and texture information. Specifically, two cross-attention-based fusion layers are proposed to fuse the texture information from paired images and structure information from point correspondences. Moreover, we propose a convolutional position encoding layer to enhance the difference between Tokens and enable the encoding feature pay attention to neighbor information. Our position encoding layer will make the cross-attention operation integrate both local and global information. Experiments on multiple datasets(3DMatch, 3DLoMatch, KITTI) and recent state-of-the-art models (3DRegNet, DGR, PointDSC) prove that our GMF achieves wide generalization ability and consistently improves the point cloud registration accuracy. Furthermore, several ablation studies demonstrate the robustness of the proposed GMF on different loss functions, lighting conditions and noises.The code is available at https://github.com/XiaoshuiHuang/GMF.
翻译:拒绝信件外端的通用多模式聚合(GMF) 能够提高信件外端的质量, 这是实现高点云层登记准确性的关键步骤 。 目前最先进的信件外端拒绝方法仅使用信件的结构特征 。 然而, 纹理信息对于拒绝我们人类视觉系统中的通信外端至关重要 。 在本文中, 我们提议通用多模式融合( GMF) 来学习如何通过利用结构和纹理信息来拒绝信件外端 。 具体地说, 提议了两个基于交叉注意的聚合层, 以整合来自配对图像的纹理信息以及点通信中的结构信息 。 此外, 我们提议了一个革命性位置编码层, 以强化 Tokens 之间的差异, 并启用编码特性来吸引邻居信息 。 我们的位置编码层将使交叉注意操作将本地和全球信息结合起来。 对多个数据集( 3DMDMatch, 3DLOMT, KITTI) 进行实验, 以及最近的州- 艺术模型( 3DregNet, DGR, PentDSC) 证明我们的GF 能够实现广度通用/ Climationalationality 的功能, 。 以及不断提高 的 Climationalationality 。