基于多尺度时空注意力的上下文感知网络用于视频行为识别 (Context-Aware Network Based on Multi-scale Spatio-temporal Attention for Action Recognition in Videos)

Action recognition is a critical task in video understanding, requiring the comprehensive capture of spatio-temporal cues across various scales. However, existing methods often overlook the multi-granularity nature of actions. To address this limitation, we introduce the Context-Aware Network (CAN). CAN consists of two core modules: the Multi-scale Temporal Cue Module (MTCM) and the Group Spatial Cue Module (GSCM). MTCM effectively extracts temporal cues at multiple scales, capturing both fast-changing motion details and overall action flow. GSCM, on the other hand, extracts spatial cues at different scales by grouping feature maps and applying specialized extraction methods to each group. Experiments conducted on five benchmark datasets (Something-Something V1 and V2, Diving48, Kinetics-400, and UCF101) demonstrate the effectiveness of CAN. Our approach achieves competitive performance, outperforming most mainstream methods, with accuracies of 50.4% on Something-Something V1, 63.9% on Something-Something V2, 88.4% on Diving48, 74.9% on Kinetics-400, and 86.9% on UCF101. These results highlight the importance of capturing multi-scale spatio-temporal cues for robust action recognition.

翻译：行为识别是视频理解中的关键任务，需要全面捕捉不同尺度的时空线索。然而，现有方法往往忽视了行为的多粒度特性。为解决这一局限，我们提出了上下文感知网络（CAN）。CAN包含两个核心模块：多尺度时序线索模块（MTCM）和分组空间线索模块（GSCM）。MTCM能有效提取多尺度时序线索，同时捕捉快速变化的运动细节和整体行为流。另一方面，GSCM通过对特征图进行分组并对每组应用专门的提取方法，提取不同尺度的空间线索。在五个基准数据集（Something-Something V1和V2、Diving48、Kinetics-400和UCF101）上进行的实验验证了CAN的有效性。我们的方法取得了具有竞争力的性能，优于大多数主流方法，在Something-Something V1上准确率达到50.4%，在Something-Something V2上达到63.9%，在Diving48上达到88.4%，在Kinetics-400上达到74.9%，在UCF101上达到86.9%。这些结果凸显了捕捉多尺度时空线索对于鲁棒行为识别的重要性。