Recent multi-camera 3D object detectors usually leverage temporal information to construct multi-view stereo that alleviates the ill-posed depth estimation. However, they typically assume all the objects are static and directly aggregate features across frames. This work begins with a theoretical and empirical analysis to reveal that ignoring the motion of moving objects can result in serious localization bias. Therefore, we propose to model Dynamic Objects in RecurrenT (DORT) to tackle this problem. In contrast to previous global Bird-Eye-View (BEV) methods, DORT extracts object-wise local volumes for motion estimation that also alleviates the heavy computational burden. By iteratively refining the estimated object motion and location, the preceding features can be precisely aggregated to the current frame to mitigate the aforementioned adverse effects. The simple framework has two significant appealing properties. It is flexible and practical that can be plugged into most camera-based 3D object detectors. As there are predictions of object motion in the loop, it can easily track objects across frames according to their nearest center distances. Without bells and whistles, DORT outperforms all the previous methods on the nuScenes detection and tracking benchmarks with 62.5\% NDS and 57.6\% AMOTA, respectively. The source code will be released.
翻译:最近的多相机三维物体检测器通常利用时间信息构建多视图立体,并减轻不良的深度估计。然而,它们通常假设所有对象都是静态的,并直接跨帧聚合特征。本文从理论和实证分析的角度入手,揭示了忽略运动物体的运动可能导致严重的定位偏差。因此,为解决这个问题,我们提出了Dynamic Objects in RecurrenT(DORT)模型。与以往的全局鸟瞰方法不同,DORT提取适用于运动估计的物体局部卷积,也减轻了复杂的计算负担。通过迭代细化估计的物体运动和位置,可以将前面的特征精确地聚合到当前帧,以缓解上述负面影响。这个简单的框架有两个显著有吸引力的属性。它是灵活实用的,可以插入到大多数基于相机的3D物体检测器中。由于循环中存在物体运动的预测,因此可以根据最近的中心距离轻松跟踪物体。在没有花哨的情况下,DORT在nuScenes检测和跟踪基准上表现优异,分别达到了62.5%的NDS和57.6%的AMOTA。源代码将会开放。