This paper presents a novel architecture for simultaneous estimation of highly accurate optical flows and rigid scene transformations for difficult scenarios where the brightness assumption is violated by strong shading changes. In the case of rotating objects or moving light sources, such as those encountered for driving cars in the dark, the scene appearance often changes significantly from one view to the next. Unfortunately, standard methods for calculating optical flows or poses are based on the expectation that the appearance of features in the scene remain constant between views. These methods may fail frequently in the investigated cases. The presented method fuses texture and geometry information by combining image, vertex and normal data to compute an illumination-invariant optical flow. By using a coarse-to-fine strategy, globally anchored optical flows are learned, reducing the impact of erroneous shading-based pseudo-correspondences. Based on the learned optical flows, a second architecture is proposed that predicts robust rigid transformations from the warped vertex and normal maps. Particular attention is payed to situations with strong rotations, which often cause such shading changes. Therefore a 3-step procedure is proposed that profitably exploits correlations between the normals and vertices. The method has been evaluated on a newly created dataset containing both synthetic and real data with strong rotations and shading effects. This data represents the typical use case in 3D reconstruction, where the object often rotates in large steps between the partial reconstructions. Additionally, we apply the method to the well-known Kitti Odometry dataset. Even if, due to fulfillment of the brighness assumption, this is not the typical use case of the method, the applicability to standard situations and the relation to other methods is therefore established.
翻译:本文为同时估计高度精确的光学流和僵硬的场景变化提供了一个新结构,用于同时估计高度精确的光学流和对亮度假设因强烈的阴影变化而违反的艰难情景的僵硬场景变化。 在旋转对象或移动光源,如在黑暗中驾驶汽车遇到的物体时,场景的外观往往从一个角度变化到下一个角度。 不幸的是,计算光学流或表面表现的标准方法基于一种期望,即场景特征的外观在不同观点之间保持恒定不变。在调查的案件中,这些方法可能经常失败。 所提出的方法将光度和几何光度信息结合成质和几何性信息,将图像、顶部和正常数据组合合并,以计算出正向偏差的偏差部分光学流。因此,通过使用粗向偏差的光学战略,全球定位光学流动,减少了错误的阴影光学流的影响。 根据所学的光学流,第二个结构可以预测从扭曲的脊椎和普通地图的僵硬的变换方法, 特别是由于图像的旋转而导致的情况,往往会改变透视正正反的偏向。 因此,在正常的周期中,在正常的变换法的变换法中, 假设中, 将数据法的变的变换法的变的变的变的变的变的变的变的变的变的变的变法是, 变的变法是新法是新的数据法是新的数据法,,, 变法是新的变的变法是新的变的变的变的变的变的变的变的变的变的变的变的变法 。 。在的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变法是的变法是的变法是的变法是的变的变的变的变的变的变的变的变法是的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变的变法,, 。