Unprecedented access to multi-temporal satellite imagery has opened new perspectives for a variety of Earth observation tasks. Among them, pixel-precise panoptic segmentation of agricultural parcels has major economic and environmental implications. While researchers have explored this problem for single images, we argue that the complex temporal patterns of crop phenology are better addressed with temporal sequences of images. In this paper, we present the first end-to-end, single-stage method for panoptic segmentation of Satellite Image Time Series (SITS). This module can be combined with our novel image sequence encoding network which relies on temporal self-attention to extract rich and adaptive multi-scale spatio-temporal features. We also introduce PASTIS, the first open-access SITS dataset with panoptic annotations. We demonstrate the superiority of our encoder for semantic segmentation against multiple competing architectures, and set up the first state-of-the-art of panoptic segmentation of SITS. Our implementation and PASTIS are publicly available.
翻译:获得多时卫星图像的前所未有的机会为各种地球观测任务开辟了新的视角,其中包括:对农业区块进行像素精细泛光分割具有重大的经济和环境影响。研究人员探讨了单一图像的这一问题,但我们认为,作物个性学复杂的时间模式最好通过图像的时间序列来解决。我们在本文件中介绍了第一个端到端、单阶段的卫星图像时间序列截面分割法(SITS)。这个模块可以与我们的新颖图像序列编码网络相结合,这个网络依靠时间的自我意识来提取丰富和适应性强的多尺度时空特征。我们还引入了PASTIS,这是第一个带有全光学说明的开放访问SITS数据集。我们展示了我们的编码器在对多个相竞争的结构进行语义分割的优越性,并设置了SITS的首个光谱分割状态。我们的实施和PASTIS是公开的。