As an essential processing step before the fusing of infrared and visible images, the performance of image registration determines whether the two images can be fused at correct spatial position. In the actual scenario, the varied imaging devices may lead to a change in perspective or time gap between shots, making significant non-rigid spatial relationship in infrared and visible images. Even if a large number of feature points are matched, the registration accuracy may still be inadequate, affecting the result of image fusion and other vision tasks. To alleviate this problem, we propose a Semantic-Aware on-Demand registration network (SA-DNet), which mainly purpose is to confine the feature matching process to the semantic region of interest (sROI) by designing semantic-aware module (SAM) and HOL-Deep hybrid matching module (HDM). After utilizing TPS to transform infrared and visible images based on the corresponding feature points in sROI, the registered images are fused using image fusion module (IFM) to achieve a fully functional registration and fusion network. Moreover, we point out that for different demands, this type of approach allows us to select semantic objects for feature matching as needed and accomplishes task-specific registration based on specific requirements. To demonstrate the robustness of SA-DNet for non-rigid distortions, we conduct extensive experiments by comparing SA-DNet with five state-of-the-art infrared and visible image feature matching methods, and the experimental results show that our method adapts better to the presence of non-rigid distortions in the images and provides semantically well-registered images.
翻译:作为红外和可见图像引信之前的一个基本处理步骤,图像注册的性能决定了两种图像能否在正确的空间位置上结合。在实际情景中,不同的成像装置可能导致镜头之间视野或时间间隔的改变,使红红外图像和可见图像之间具有重要的非硬性空间关系。即使大量特征点相匹配,登记准确性可能仍然不够充分,影响到图像聚合和其他视觉任务的结果。为了缓解这一问题,我们提议建立一个Sermantitic-Aware in-Deman登记网络(SA-DNet)网络(SA-DNet),其主要目的是通过设计语义识别模块(SAM)和HOL-Deep 混合匹配模块(HDM),将功能匹配进程限制在关注的语义区域(sROI)上。在利用TPS转换红外和可见图像后,注册图像将使用图像聚合模块(IFM)实现完全功能的注册和聚合网络注册网络网络网络网络(SA-D)网络(SA-D)网络(SA-D) 网络(SL) 的清晰性图像变校正的图像变校正化方法使得我们能够选择精度的精确的图像变校正的精确化方法,从而显示我们所需的具体的图像的图像的精确化,从而显示特性的精确化方法,从而显示的特性的特性的特性的特性,从而显示我们在特定的特性上显示的特性的精确化的特性的特性,从而显示的特性的特性的特性的特性的特性的特性的特性,从而展示的精确化的特性的特性的特性的特性的特性,从而显示我们选择。