Vision transformers are emerging as a powerful tool to solve computer vision problems. Recent techniques have also proven the efficacy of transformers beyond the image domain to solve numerous video-related tasks. Among those, human action recognition is receiving special attention from the research community due to its widespread applications. This article provides the first comprehensive survey of vision transformer techniques for action recognition. We analyze and summarize the existing and emerging literature in this direction while highlighting the popular trends in adapting transformers for action recognition. Due to their specialized application, we collectively refer to these methods as ``action transformers''. Our literature review provides suitable taxonomies for action transformers based on their architecture, modality, and intended objective. Within the context of action transformers, we explore the techniques to encode spatio-temporal data, dimensionality reduction, frame patch and spatio-temporal cube construction, and various representation methods. We also investigate the optimization of spatio-temporal attention in transformer layers to handle longer sequences, typically by reducing the number of tokens in a single attention operation. Moreover, we also investigate different network learning strategies, such as self-supervised and zero-shot learning, along with their associated losses for transformer-based action recognition. This survey also summarizes the progress towards gaining grounds on evaluation metric scores on important benchmarks with action transformers. Finally, it provides a discussion on the challenges, outlook, and future avenues for this research direction.
翻译:视觉变异器正在成为解决计算机视觉问题的有力工具。最近的技术也证明变异器在图像领域外的变异器在解决众多视频相关任务方面的功效。其中,人类行动识别因其广泛应用而得到研究界的特别关注。本文章首次对视觉变异器技术进行了综合调查,供行动识别使用。我们分析和总结了这方面的现有和新兴文献,同时着重介绍了使变异器适应行动识别的流行趋势。由于这些方法的专门应用,我们将这些方法统称为“行动变异器”。我们文献审查根据其结构、模式和预定目标为行动变异器提供了适当的分类。在行动变异器的背景下,我们探索了将Spotio-时间数据编码、维度减少、框架补丁和时尚立立立立立器技术进行首次全面调查的技术,以及各种代表方法。我们还调查了变异器层对更长时间序列的关注,通常通过在单一关注操作中减少标志的数量。此外,我们还调查了不同的网络学习战略,例如:最终的变异式研究方向、最终的变异式评估,以及最终的进度评估。