Hyperspectral images (HSI) provide rich spectral information that contributed to the successful performance improvement of numerous computer vision tasks. However, it can only be achieved at the expense of images' spatial resolution. Hyperspectral image super-resolution (HSI-SR) addresses this problem by fusing low resolution (LR) HSI with multispectral image (MSI) carrying much higher spatial resolution (HR). All existing HSI-SR approaches require the LR HSI and HR MSI to be well registered and the reconstruction accuracy of the HR HSI relies heavily on the registration accuracy of different modalities. This paper exploits the uncharted problem domain of HSI-SR without the requirement of multi-modality registration. Given the unregistered LR HSI and HR MSI with overlapped regions, we design a unique unsupervised learning structure linking the two unregistered modalities by projecting them into the same statistical space through the same encoder. The mutual information (MI) is further adopted to capture the non-linear statistical dependencies between the representations from two modalities (carrying spatial information) and their raw inputs. By maximizing the MI, spatial correlations between different modalities can be well characterized to further reduce the spectral distortion. A collaborative $l_{2,1}$ norm is employed as the reconstruction error instead of the more common $l_2$ norm, so that individual pixels can be recovered as accurately as possible. With this design, the network allows to extract correlated spectral and spatial information from unregistered images that better preserves the spectral information. The proposed method is referred to as unregistered and unsupervised mutual Dirichlet Net ($u^2$-MDN). Extensive experimental results using benchmark HSI datasets demonstrate the superior performance of $u^2$-MDN as compared to the state-of-the-art.
翻译:超光谱图像(HSI)提供丰富的光谱信息,有助于成功改进许多计算机视觉任务的业绩;然而,只有牺牲图像的空间分辨率,才能做到这一点。超光谱图像超分辨率(HSI-SR)通过使用低分辨率(LR)HSI和多光谱图像(MSI)来解决这个问题。所有现有的HSI-SR方法都要求LR HSI和HR MSI得到良好的登记,而HR HSI的重建准确性在很大程度上取决于不同模式的登记准确性。本文利用了HSI-SR的未知问题域,而没有采用多模式登记。鉴于未注册的LRHSI和HR MSI与重叠区域之间的未注册分辨率(HSI),我们设计了一个独特的、不超高的学习结构,通过同一编码将两种未注册模式投放到同一统计空间。 相互信息(MI)被进一步采用,从两种模式(记录空间信息)到它们的原始输入。通过最大程度的MI、空间-SLSI的高级数据(DRI) 和Siral-rationalal 数据,可以将共同的精确性标准(Orationalalalalalalal)转化为。