Multi-camera surveillance has been an active research topic for understanding and modeling scenes. Compared to a single camera, multi-cameras provide larger field-of-view and more object cues, and the related applications are multi-view counting, multi-view tracking, 3D pose estimation or 3D reconstruction, etc. It is usually assumed that the cameras are all temporally synchronized when designing models for these multi-camera based tasks. However, this assumption is not always valid,especially for multi-camera systems with network transmission delay and low frame-rates due to limited network bandwidth, resulting in desynchronization of the captured frames across cameras. To handle the issue of unsynchronized multi-cameras, in this paper, we propose a synchronization model that works in conjunction with existing DNN-based multi-view models, thus avoiding the redesign of the whole model. Under the low-fps regime, we assume that only a single relevant frame is available from each view, and synchronization is achieved by matching together image contents guided by epipolar geometry. We consider two variants of the model, based on where in the pipeline the synchronization occurs, scene-level synchronization and camera-level synchronization. The view synchronization step and the task-specific view fusion and prediction step are unified in the same framework and trained in an end-to-end fashion. Our view synchronization models are applied to different DNNs-based multi-camera vision tasks under the unsynchronized setting, including multi-view counting and 3D pose estimation, and achieve good performance compared to baselines.
翻译:多相机监视一直是了解和建模场景的积极研究主题。 与单一相机相比, 多相机提供了更大的视野和更多的对象提示,相关应用程序是多视图计数、多视图跟踪、 3D 显示估计或3D重建等。 通常假设在设计这些基于多相机的任务模型时,相机都具有时间同步性。 然而, 这一假设并不总是有效的, 特别是对于拥有网络传输延迟和多框架率低的多相机系统, 其原因是网络带宽有限, 从而导致摄像头之间捕获的框脱同步化。 为了处理不同步多摄像头问题, 在本文件中, 我们提出一个同步模型, 与现有的基于DNNN的多视图模型一起工作, 从而避免对整个模型进行重新设计。 在低频系统下, 我们假设, 只能从每种视图中找到一个单一的相关框架, 并且通过以直径直线计算为指南的图像内容相匹配, 从而实现同步。 我们考虑两种级别模型的变式模型, 其基础是:在管道中, 同步、 同步化到同步度下, 同步度下级和级的同步定位框架下, 级, 级, 级, 级, 级, 级, 级和级级级的同步定位为同一级, 级, 级, 级, 级级级级, 级级和级级级级级级级级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级定级, 级, 级和级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级, 级,