Many classical and learning-based optical flow methods rely on hierarchical concepts to improve both accuracy and robustness. However, one of the currently most successful approaches -- RAFT -- hardly exploits such concepts. In this work, we show that multi-scale ideas are still valuable. More precisely, using RAFT as a baseline, we propose a novel multi-scale neural network that combines several hierarchical concepts within a single estimation framework. These concepts include (i) a partially shared coarse-to-fine architecture, (ii) multi-scale features, (iii) a hierarchical cost volume and (iv) a multi-scale multi-iteration loss. Experiments on MPI Sintel and KITTI clearly demonstrate the benefits of our approach. They show not only substantial improvements compared to RAFT, but also state-of-the-art results -- in particular in non-occluded regions. Code will be available at https://github.com/cv-stuttgart/MS_RAFT.
翻译:许多古典和以学习为基础的光学流动方法依靠等级概念来提高准确性和稳健性。然而,目前最成功的方法之一 -- -- RAFT -- -- 几乎没有利用这些概念。我们在此工作中表明,多规模的想法仍然很宝贵。更准确地说,我们提议采用RAFT作为基准,建立一个新的多规模神经网络,将几个等级概念结合到一个单一的估计框架内。这些概念包括:(一) 部分共享的粗皮到纤维结构,(二) 多尺度特征,(三) 等级成本量,(四) 多尺度的多功能损失。关于MPI Sintel和KITTI的实验清楚地表明了我们的方法的好处。它们不仅表明与RAFTT相比有了重大改进,而且显示了最新的结果 -- -- 特别是在非隐蔽地区。守则将在https://github.com/cv-stutgart/MS_RAFT上查阅。