Temporal action proposal generation aims to estimate temporal intervals of actions in untrimmed videos, which is a challenging yet important task in the video understanding field. The proposals generated by current methods still suffer from inaccurate temporal boundaries and inferior confidence used for retrieval owing to the lack of efficient temporal modeling and effective boundary context utilization. In this paper, we propose Temporal Context Aggregation Network (TCANet) to generate high-quality action proposals through "local and global" temporal context aggregation and complementary as well as progressive boundary refinement. Specifically, we first design a Local-Global Temporal Encoder (LGTE), which adopts the channel grouping strategy to efficiently encode both "local and global" temporal inter-dependencies. Furthermore, both the boundary and internal context of proposals are adopted for frame-level and segment-level boundary regressions, respectively. Temporal Boundary Regressor (TBR) is designed to combine these two regression granularities in an end-to-end fashion, which achieves the precise boundaries and reliable confidence of proposals through progressive refinement. Extensive experiments are conducted on three challenging datasets: HACS, ActivityNet-v1.3, and THUMOS-14, where TCANet can generate proposals with high precision and recall. By combining with the existing action classifier, TCANet can obtain remarkable temporal action detection performance compared with other methods. Not surprisingly, the proposed TCANet won the 1$^{st}$ place in the CVPR 2020 - HACS challenge leaderboard on temporal action localization task.
翻译:时间行动提案的生成旨在估计未剪辑的视频中行动的时间间隔,这是视频理解领域一项具有挑战性但重要的任务。目前方法产生的建议仍然由于时间边界不准确,而且由于缺乏高效的时间模型和有效利用边界背景,检索时地环境集合网络(TTCANet)缺乏高效的时间模型和有效利用,因此,在本文中,我们提议时地环境集合网络(TCANet)通过“当地和全球”时间背景汇总和补充以及逐步完善边界,产生高质量的行动提案。具体地说,我们首先设计了一个地方-全球时地摄像仪(LGTE),采用频道组合战略,高效率地对“当地和全球”的时间相互依存关系进行编码。此外,在框架一级和部分一级边界回归方面,我们采纳了提案的边界和内部背景背景。