The abundance of gaps in satellite image time series often complicates the application of deep learning models such as convolutional neural networks for spatiotemporal modeling. Based on previous work in computer vision on image inpainting, this paper shows how three-dimensional spatiotemporal partial convolutions can be used as layers in neural networks to fill gaps in satellite image time series. To evaluate the approach, we apply a U-Net-like model on incomplete image time series of quasi-global carbon monoxide observations from the Sentinel-5P satellite. Prediction errors were comparable to two considered statistical approaches while computation times for predictions were up to three orders of magnitude faster, making the approach applicable to process large amounts of satellite data. Partial convolutions can be added as layers to other types of neural networks, making it relatively easy to integrate with existing deep learning models. However, the approach does not quantify prediction errors and further research is needed to understand and improve model transferability. The implementation of spatiotemporal partial convolutions and the U-Net-like model is available as open-source software.
翻译:卫星图象时间序列中的大量空白往往使深层学习模型的应用复杂化,例如用于时空建模的进化神经网络等深层学习模型的应用复杂化。根据计算机图像成像图的计算机视觉先前工作,本文件展示了三维空间部分演化如何可以用作神经网络中的层以填补卫星图像时间序列中的空白。为了评估这一方法,我们应用了一个类似U-Net的模型,用于半全球一氧化碳观测的不完整图像时间序列。预测错误可与两种经过考虑的统计方法相比,而预测的计算时间可更快地达到三个数量级,使该方法适用于处理大量卫星数据。部分演化可以作为其他类型的神经网络的层而增加,使之相对容易与现有的深层学习模型融合。然而,这一方法并不量化预测错误,需要进一步研究才能理解和改进模型的可转移性。可以将空间部分演化和U-Net的模型作为开放源软件使用。