Multispectral image fusion is a computer vision process that is essential to remote sensing. For applications such as dehazing and object detection, there is a need to offer solutions that can perform in real-time on any type of scene. Unfortunately, current state-of-the-art approaches do not meet these criteria as they need to be trained on domain-specific data and have high computational complexity. This paper focuses on the task of fusing color (RGB) and near-infrared (NIR) images as this the typical RGBT sensors, as in multispectral cameras for detection, fusion, and dehazing. Indeed, the NIR channel has the ability to capture details not visible in RGB and see beyond haze, fog, and clouds. To combine this information, a novel approach based on superpixel segmentation is designed so that multispectral image fusion is performed according to the specific local content of the images to be fused. Therefore, the proposed method produces a fusion that contains the most relevant content of each spectrum. The experiments reported in this manuscript show that the novel approach better preserve details than alternative fusion methods.
翻译:多光谱图像聚合是一个对遥感至关重要的计算机视觉过程。 对于拆解和物体探测等应用,需要提供能够在任何类型的场景上实时运行的解决方案。 不幸的是,目前最先进的方法并不满足这些标准,因为它们需要接受特定领域数据的培训,并且具有很高的计算复杂性。本文侧重于引信颜色(RGB)和近红外(NIR)图像的任务,作为典型的RGBT传感器,如用于探测、聚变和拆解的多光谱照相机。事实上,NIR频道有能力捕捉在RGB中看不到的细节,并能够看到烟雾、雾和云层之外。为了将这一信息结合起来,设计了一个基于超像素分层的新办法,以便根据要结合的图像的具体本地内容进行多光谱图像聚合。因此,拟议的方法产生一个包含每个频谱最相关内容的聚变器。本中报道的实验显示,新办法比替代的聚变法更好地保存细节。