Inferring geometrically consistent dense 3D scenes across a tuple of temporally consecutive images remains challenging for self-supervised monocular depth prediction pipelines. This paper explores how the increasingly popular transformer architecture, together with novel regularized loss formulations, can improve depth consistency while preserving accuracy. We propose a spatial attention module that correlates coarse depth predictions to aggregate local geometric information. A novel temporal attention mechanism further processes the local geometric information in a global context across consecutive images. Additionally, we introduce geometric constraints between frames regularized by photometric cycle consistency. By combining our proposed regularization and the novel spatial-temporal-attention module we fully leverage both the geometric and appearance-based consistency across monocular frames. This yields geometrically meaningful attention and improves temporal depth stability and accuracy compared to previous methods.
翻译:本文探讨日益流行的变压器结构,加上新型的常规损失配方,如何在保持准确性的同时提高深度一致性。我们提议了一个空间关注模块,将粗微的深度预测与汇总当地几何信息联系起来。一个新的时间关注机制进一步在全球范围内处理连续图像中的当地几何信息。此外,我们引入了以光度周期一致性规范化的框架之间的几何限制。通过将我们提议的正规化和新的空间时空注意模块结合起来,我们充分利用了单眼框架的几何和外观一致性。这产生了几何上有意义的关注,提高了与以往方法相比的时间深度稳定性和准确性。