Self-supervised learning techniques are gaining popularity due to their capability of building models that are effective, even when scarce amounts of labeled data are available. In this paper, we present a framework and specific tasks for self-supervised training of multichannel models, such as the fusion of multispectral and synthetic aperture radar images. We show that the proposed self-supervised approach is highly effective at learning features that correlate with the labels for land cover classification. This is enabled by an explicit design of pretraining tasks which promotes bridging the gaps between sensing modalities and exploiting the spectral characteristics of the input. When limited labels are available, using the proposed self-supervised pretraining and supervised finetuning for land cover classification with SAR and multispectral data outperforms conventional approaches such as purely supervised learning, initialization from training on Imagenet and recent self-supervised approaches for computer vision tasks.
翻译:自我监督的学习技术由于建立有效模型的能力而越来越受欢迎,即使有很少的标签数据,在本文件中,我们提出了一个框架和具体任务,用于对多通道模型进行自我监督的培训,例如将多光谱和合成孔径雷达图像融合在一起。我们表明,拟议的自我监督方法在学习与土地覆盖分类标签相关的特征方面非常有效。这通过明确设计培训前任务而得以实现,这种培训任务有助于缩小遥感模式与利用输入的光谱特征之间的差距。如果存在有限的标签,则使用拟议的自我监督的预先培训和监督的对土地覆盖分类进行微调,使用搜索和合成孔径雷达图像和多光谱数据超越常规方法,如纯监督的学习、通过图像网培训的初始化和最近计算机视觉任务自监督的方法。