In this paper, we propose an efficient and generalizable framework based on deep convolutional neural network (CNN) for multi-source remote sensing data joint classification. While recent methods are mostly based on multi-stream architectures, we use group convolution to construct equivalent network architectures efficiently within a single-stream network. We further adopt and improve dynamic grouping convolution (DGConv) to make group convolution hyperparameters, and thus the overall network architecture, learnable during network training. The proposed method therefore can theoretically adjust any modern CNN models to any multi-source remote sensing data set, and can potentially avoid sub-optimal solutions caused by manually decided architecture hyperparameters. In the experiments, the proposed method is applied to ResNet and UNet, and the adjusted networks are verified on three very diverse benchmark data sets (i.e., Houston2018 data, Berlin data, and MUUFL data). Experimental results demonstrate the effectiveness of the proposed single-stream CNNs, and in particular ResNet18-DGConv improves the state-of-the-art classification overall accuracy (OA) on HS-SAR Berlin data set from $62.23\%$ to $68.21\%$. In the experiments we have two interesting findings. First, using DGConv generally reduces test OA variance. Second, multi-stream is harmful to model performance if imposed to the first few layers, but becomes beneficial if applied to deeper layers. Altogether, the findings imply that multi-stream architecture, instead of being a strictly necessary component in deep learning models for multi-source remote sensing data, essentially plays the role of model regularizer. Our code is publicly available at https://github.com/yyyyangyi/Multi-source-RS-DGConv. We hope our work can inspire novel research in the future.
翻译:在本文中,我们提出了一个基于深层神经神经神经网络(CNN)的高效和可概括的框架,用于多源遥感数据联合分类。虽然最近的方法大多以多流结构为基础,但我们使用集团革命在单一流网络中高效地构建等效网络架构。我们进一步采用和改进动态组合(DGConv),使集团进化超参数,从而形成整体网络架构,在网络培训期间可以学习。因此,拟议方法可以在理论上将任何现代CNN模型调整为任何多源遥感数据集,并有可能避免人工决定的架构的超参数造成的亚最佳解决方案。在实验中,拟议方法将适用于ResNet和UNet,调整后的网络将在三个非常多样化的基准数据集(即休斯顿2018数据、柏林数据和穆林尔福尔夫勒数据)上进行校验。实验结果显示,拟议的单流模式CNNW,特别是ResNet18-DGCon,可以改进高级分类的多源遥感数据集总体精确度(OA),在HS-SAR-SO-SOOOOO-DO的高级测试结果中,在SAR-DG-DG-DO-DVAL的常规数据中,在SD-D-D-DOralal Stalal Stalal Stal Stal Studal Studal Studal Studal Stutyal Stutyal Stutyal Stal Stutexxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx