Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral tradeoff. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from an RGB image. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep-learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral tradeoff, offering simple hardware requirements and potential applications of various machine-learning techniques.
翻译:高光谱成像获取的数据既包含空间信息又包含频率信息,因此能够提供大量的物理或生物信息。然而,传统的高光谱成像具有体积庞大、数据采集速率缓慢和空谱折衷等固有局限性。本文介绍了一种用于快照高光谱成像的高光谱学习方法,将从一个小子区域中采样的高光谱数据融合到学习算法中,以恢复超立方体。高光谱学习利用了一个观点,即照片不仅仅是一张图片,还包含了详细的光谱信息。小样本的高光谱数据使得光谱感知学习能够从RGB图像中恢复超立方体。高光谱学习能够恢复超立方体的全光谱分辨率,与科学光谱仪的高光谱分辨率相媲美。高光谱学习还能够实现超快速的动态成像,利用一个视频来录制多个RGB图像的时间序列。为了证明其多样性,我们采用了外周血管发展的实验模型,利用统计和深度学习方法提取血液动力学参数。随后,利用普通的智能手机相机,以高达毫秒级的超快速时间分辨率,评估了外周微循环的血液动力学状态。这种光谱感知学习方法类似于压缩感知技术,但它不仅允许可靠的超立方体恢复和关键特征提取,还具有透明的学习算法。此学习驱动的快照高光谱成像方法具有高光谱分辨率和时间分辨率较高的特点,并消除了空谱折衷的问题,提供了简单的硬件要求以及各种机器学习技术的潜在应用。