Video action recognition is one of the representative tasks for video understanding. Over the last decade, we have witnessed great advancements in video action recognition thanks to the emergence of deep learning. But we also encountered new challenges, including modeling long-range temporal information in videos, high computation costs, and incomparable results due to datasets and evaluation protocol variances. In this paper, we provide a comprehensive survey of over 200 existing papers on deep learning for video action recognition. We first introduce the 17 video action recognition datasets that influenced the design of models. Then we present video action recognition models in chronological order: starting with early attempts at adapting deep learning, then to the two-stream networks, followed by the adoption of 3D convolutional kernels, and finally to the recent compute-efficient models. In addition, we benchmark popular methods on several representative datasets and release code for reproducibility. In the end, we discuss open problems and shed light on opportunities for video action recognition to facilitate new research ideas.
翻译：视频行动识别是具有代表性的视频理解任务之一。 在过去的十年中,由于深层学习的出现,我们在视频行动识别方面取得了巨大进步。但我们也遇到了新的挑战,包括视频中模拟远程时间信息、高计算成本以及由于数据集和评价协议差异而导致的无法比较的结果。在本文中,我们对200多份关于深层学习用于视频行动识别的现有论文进行了全面调查。我们首先介绍了影响模型设计的17个视频行动识别数据集。然后,我们按时间顺序展示视频行动识别模型:先是尝试对深层学习进行早期调整,然后是进入双流网络,然后是采用3D革命核心,最后是最近的计算效率模型。此外,我们还将流行方法以若干具有代表性的数据集和发布代码作为基准,以便重新展示。最后,我们讨论了公开的问题,并介绍了视频行动识别机会,以促进新的研究想法。
相关内容
微信扫码咨询专知VIP会员