Visual representation learning is ubiquitous in various real-world applications, including visual comprehension, video understanding, multi-modal analysis, human-computer interaction, and urban computing. Due to the emergence of huge amounts of multi-modal heterogeneous spatial/temporal/spatial-temporal data in big data era, the lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. The majority of the existing methods tend to fit the original data/variable distributions and ignore the essential causal relations behind the multi-modal knowledge, which lacks unified guidance and analysis about why modern visual representation learning methods easily collapse into data bias and have limited generalization and cognitive abilities. Inspired by the strong inference ability of human-level agents, recent years have therefore witnessed great effort in developing causal reasoning paradigms to realize robust representation and model learning with good cognitive ability. In this paper, we conduct a comprehensive review of existing causal reasoning methods for visual representation learning, covering fundamental theories, models, and datasets. The limitations of current methods and datasets are also discussed. Moreover, we propose some prospective challenges, opportunities, and future research directions for benchmarking causal reasoning algorithms in visual representation learning. This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods, publicly available benchmarks, and consensus-building standards for reliable visual representation learning and related real-world applications more efficiently.
翻译:视觉表示学习是各种现实应用中普遍存在的,包括视觉理解、视频理解、多模态分析、人机交互和城市计算等。由于大数据时代出现了大量的多模态异构空间/时间/空间-时间数据,现有视觉模型的缺乏解释性、鲁棒性和越界泛化已经成为挑战。大多数现有的方法倾向于适应原始数据/变量分布,并忽略了多模态知识后面的本质因果关系,这缺乏统一的指导和分析,从而难以解释现代视觉表示学习方法为何容易出现数据偏差,而且具有有限的泛化和认知能力。得益于人类等价代理的强大推理能力,近几年来,已经见证了在实现具有良好认知能力的鲁棒表示和模型学习方面开发因果推理范式的巨大努力。在本文中,我们对现有视觉表示学习因果推理方法,包括基本理论、模型和数据集进行了全面的审查。同时,我们还讨论了当前方法和数据集的局限性。此外,我们提出了一些前瞻性挑战、机遇和未来研究方向,以在视觉表示学习中对因果推理算法进行基准测试。本文旨在提供这一新兴领域的全面概述,引起关注,促进讨论,突出开发新的因果推理方法,公开的基准,以及可靠的视觉表示学习和相关现实应用的共识建立标准的紧迫性。