Conventionally, Earth system (e.g., weather and climate) forecasting relies on numerical simulation with complex physical models and are hence both expensive in computation and demanding on domain expertise. With the explosive growth of the spatiotemporal Earth observation data in the past decade, data-driven models that apply Deep Learning (DL) are demonstrating impressive potential for various Earth system forecasting tasks. The Transformer as an emerging DL architecture, despite its broad success in other domains, has limited adoption in this area. In this paper, we propose Earthformer, a space-time Transformer for Earth system forecasting. Earthformer is based on a generic, flexible and efficient space-time attention block, named Cuboid Attention. The idea is to decompose the data into cuboids and apply cuboid-level self-attention in parallel. These cuboids are further connected with a collection of global vectors. We conduct experiments on the MovingMNIST dataset and a newly proposed chaotic N-body MNIST dataset to verify the effectiveness of cuboid attention and figure out the best design of Earthformer. Experiments on two real-world benchmarks about precipitation nowcasting and El Nino/Southern Oscillation (ENSO) forecasting show Earthformer achieves state-of-the-art performance.
翻译:从《公约》角度看,地球系统(例如天气和气候)的预测依赖于以复杂的物理模型进行数字模拟,因此在计算和对域专门知识的要求方面费用昂贵。随着过去十年时地地球观测数据的爆炸性增长,应用深层学习(DL)的数据驱动模型展示了各种地球系统预报任务的巨大潜力。变形器尽管在其他领域取得了广泛成功,但在这一领域的新兴DL结构的采用有限。在本文件中,我们提议以地球系统预报的时空变异器Earthex(地球变异器)作为地球系统预报的时空变异器。地球变异器基于一个通用的、灵活和高效的时空关注区,名为Cuboid 注意。其想法是将数据分解成幼类,并同时应用幼类自留。这些幼类模型与全球矢量的收集进一步相连。我们进行了关于移动和移动MNIST数据集的实验,并提议建立一个混乱的NMIST数据集,以核实幼类关注的效果,并绘制出地球变异体的最佳设计。在现实世界的两个关于降压的状态基准上进行了实验。