Many algorithms have been developed to solve the inverse problem of coded aperture snapshot spectral imaging (CASSI), i.e., recovering the 3D hyperspectral images (HSIs) from a 2D compressive measurement. In recent years, learning-based methods have demonstrated promising performance and dominated the mainstream research direction. However, existing CNN-based methods show limitations in capturing long-range dependencies and non-local self-similarity. Previous Transformer-based methods densely sample tokens, some of which are uninformative, and calculate the multi-head self-attention (MSA) between some tokens that are unrelated in content. This does not fit the spatially sparse nature of HSI signals and limits the model scalability. In this paper, we propose a novel Transformer-based method, coarse-to-fine sparse Transformer (CST), firstly embedding HSI sparsity into deep learning for HSI reconstruction. In particular, CST uses our proposed spectra-aware screening mechanism (SASM) for coarse patch selecting. Then the selected patches are fed into our customized spectra-aggregation hashing multi-head self-attention (SAH-MSA) for fine pixel clustering and self-similarity capturing. Comprehensive experiments show that our CST significantly outperforms state-of-the-art methods while requiring cheaper computational costs. The code and models will be released at https://github.com/caiyuanhao1998/MST
翻译:已经开发了许多算法来解决代码孔径快照光谱成像(CASSI)的反问题,即从2D压缩测量中恢复3D超光谱图像(HSSI),近年来,以学习为基础的方法表现出了有希望的性能并主导了主流研究方向。然而,现有的CNN方法显示在捕捉远程依赖和非本地自我相似性方面存在局限性。以前以变异器为基础的方法密集样本符号,其中一些没有信息规范,并在一些内容无关的标牌之间计算多头自留(MSA)。这不符合HSI信号的空间稀少性质,限制了模型的可缩放性。在本文中,我们提出一种新的以变异器为基础的方法,即粗到松散的变异变器(CST),首先将HSI的广度嵌入为HSI重建的深层学习。特别是,科技委使用我们提议的光谱-觉检测机制(SASM)来选择与内容无关的多符号(MSA)之间。然后,将选中的补码输入到我们定制的光谱-光谱-光谱-光谱- balbal-al-mabal-commabal mology motion mology motion motion motion motional cal motional motional rodutional cal disal romotional-motion romotionalbalbalbalbalpalpalational romotional romotionalbalbisaldal-motion rocalpalpalpalpalpalps rocaldaldalps rocation rocation sabaldaldaldaldald rocaldaldaldaldaldaldal rocaldal rocal rocal rocal rocal rocal rocal rocaldaldal rocal-modal-mod rocal rocal-modal-mod rocal-modal-modal-modal-modaldaldaldaldal rocal-mocal rocald