Understanding the flow in 3D space of sparsely sampled points between two consecutive time frames is the core stone of modern geometric-driven systems such as VR/AR, Robotics, and Autonomous driving. The lack of real, non-simulated, labeled data for this task emphasizes the importance of self- or un-supervised deep architectures. This work presents a new self-supervised training method and an architecture for the 3D scene flow estimation under occlusions. Here we show that smart multi-layer fusion between flow prediction and occlusion detection outperforms traditional architectures by a large margin for occluded and non-occluded scenarios. We report state-of-the-art results on Flyingthings3D and KITTI datasets for both the supervised and self-supervised training.
翻译:了解连续两个时间框架之间分散抽样点在三维空间的流动,这是现代几何驱动系统,如VR/AR、机器人和自主驱动系统的核心石块。 缺乏实际、非模拟、标签数据为这一任务强调了自我或不受监督的深层建筑的重要性。 这项工作为三维场景流量估算提供了一种新的自我监督培训方法和架构。 在这里,我们展示了流动预测和隔离探测之间的智能多层融合,通过隐蔽和非隐蔽情景的巨大空间,使传统结构超越了隐蔽和非隐蔽情景。 我们报告了飞航3D和KITTI数据集的最新结果,供监督和自我监督的培训使用。