Causal Reasoning with Spatial-temporal Representation Learning: A Prospective Study

Spatial-temporal representation learning is ubiquitous in various real-world applications, including visual comprehension, video understanding, multi-modal analysis, human-computer interaction, and urban computing. Due to the emergence of huge amounts of multi-modal heterogeneous spatial/temporal/spatial-temporal data in big data era, the existing visual methods rely heavily on large-scale data annotations and supervised learning to learn a powerful big model. However, the lack of interpretability, robustness, and out-of-distribution generalization are becoming the bottleneck problems of these models, which hinders the progress of interpretable and reliable artificial intelligence. The majority of the existing methods are based on correlation learning with the assumption that the data are independent and identically distributed, which lack an unified guidance and analysis about why modern spatial-temporal representation learning methods have limited interpretability and easily collapse into dataset bias. Inspired by the strong inference ability of human-level agents, recent years have therefore witnessed great effort in developing causal reasoning paradigms to realize robust representation and model learning with good interpretability. In this paper, we conduct a comprehensive review of existing causal reasoning methods for spatial-temporal representation learning, covering fundamental theories, models, and datasets. The limitations of current methods and datasets are also discussed. Moreover, we propose some primary challenges, opportunities, and future research directions for benchmarking causal reasoning algorithms in spatial-temporal representation learning.