We introduce the problem of multi-camera trajectory forecasting (MCTF), which involves predicting the trajectory of a moving object across a network of cameras. While multi-camera setups are widespread for applications such as surveillance and traffic monitoring, existing trajectory forecasting methods typically focus on single-camera trajectory forecasting (SCTF), limiting their use for such applications. Furthermore, using a single camera limits the field-of-view available, making long-term trajectory forecasting impossible. We address these shortcomings of SCTF by developing an MCTF framework that simultaneously uses all estimated relative object locations from several viewpoints and predicts the object's future location in all possible viewpoints. Our framework follows a Which-When-Where approach that predicts in which camera(s) the objects appear and when and where within the camera views they appear. To this end, we propose the concept of trajectory tensors: a new technique to encode trajectories across multiple camera views and the associated uncertainties. We develop several encoder-decoder MCTF models for trajectory tensors and present extensive experiments on our own database (comprising 600 hours of video data from 15 camera views) created particularly for the MCTF task. Results show that our trajectory tensor models outperform coordinate trajectory-based MCTF models and existing SCTF methods adapted for MCTF. Code is available from: https://github.com/olly-styles/Trajectory-Tensors
翻译:我们引入了多相机轨迹预测(MCTF)问题,这涉及到通过照相机网络预测移动物体的轨迹。虽然多相机设置在监视和交通监测等应用方面十分广泛,但现有轨迹预测方法通常侧重于单相机轨迹预测(SCTF),限制其用于此类应用。此外,我们使用单一相机限制现场视野,使长期轨迹预测无法进行。我们通过开发一个多角度同时使用所有估计相对物体位置并用所有可能的视角预测该物体未来位置的 MCTF 框架来解决该工作队的这些缺陷。我们的框架遵循一种“何时何时何地预测”方法,预测在哪些相机物体出现以及何时何地,并限制其用于此类应用。为此,我们提出了轨迹色色变概念:一种在多个摄像器视图和相关不确定性之间对轨迹跟踪进行编码的新技术。我们开发了数个基于轨迹变色器/解变色器的MTF模型,并在我们自己的数据库上进行广泛的实验(从15个摄像机视图的600小时视频数据模型中进行计算 ),特别是从现有轨迹变的SDLTFTF 模式。