Deep Kernelized Dense Geometric Matching

Geometric matching is a challenging computer vision task that involves finding correspondences between two views of a 3D scene. Dense geometric matching, i.e., finding every matching pixel, is an appealing approach due to, among other things, the capacity for sub-pixel accuracy and low-texture robustness. While previous results have shown that sparse and semi-sparse methods are better suited than dense approaches for geometry estimation, we propose a novel dense method that outperforms them. We accomplish this by formulating dense global matching as a probabilistic regression task using deep kernels, in contrast to typical correlation volume processing. Furthermore, we show that replacing local correlation with warped feature stacking in the refinement stage further boosts performance. Finally, we observe that a systematic attenuation of the model confidence improves geometry estimation results. Our full approach, $\textbf{D}$eep $\textbf{K}$ernelized Dense Geometric $\textbf{M}$atching, sets a new state-of-the-art on the competitive HPatches, YFCC100m, MegaDepth-1500, and ScanNet-1500 geometry estimation benchmarks. We provide code for all our experiments, instructions for downloading datasets, and pretrained models, at https://github.com/Parskatt/dkm