This paper proposes a hybrid synthesis method for multi-exposure image fusion taken by hand-held cameras. Motions either due to the shaky camera or caused by dynamic scenes should be compensated before any content fusion. Any misalignment can easily cause blurring/ghosting artifacts in the fused result. Our hybrid method can deal with such motions and maintain the exposure information of each input effectively. In particular, the proposed method first applies optical flow for a coarse registration, which performs well with complex non-rigid motion but produces deformations at regions with missing correspondences. The absence of correspondences is due to the occlusions of scene parallax or the moving contents. To correct such error registration, we segment images into superpixels and identify problematic alignments based on each superpixel, which is further aligned by PatchMatch. The method combines the efficiency of optical flow and the accuracy of PatchMatch. After PatchMatch correction, we obtain a fully aligned image stack that facilitates a high-quality fusion that is free from blurring/ghosting artifacts. We compare our method with existing fusion algorithms on various challenging examples, including the static/dynamic, the indoor/outdoor and the daytime/nighttime scenes. Experiment results demonstrate the effectiveness and robustness of our method.
翻译:本文提出了一种用于手持相机拍摄的多曝光图像融合的混合合成方法。在进行任何内容融合之前,必须补偿由于摇晃相机或由于动态场景而引起的运动。任何不对齐都会轻易地导致融合结果中出现模糊或鬼影伪影。我们的混合方法可以处理这种运动,并有效地保留每个输入的曝光信息。特别是,所提出的方法首先应用光流进行粗略配准,可以处理复杂的非刚性运动,但在具有缺少对应关系的区域会产生变形。缺少对应关系是由于场景视差或移动内容的遮挡引起的。为了校正这种错误的配准,我们将图像分割成超像素,并根据每个超像素识别出有问题的对齐方式,并通过PatchMatch进行进一步的对齐。该方法结合了光流的效率和PatchMatch的准确性。在PatchMatch校正之后,我们获得了一个完全对齐的图像堆栈,可以实现高质量的融合,并且不会出现模糊或鬼影伪影。我们将我们的方法与现有的融合算法进行了比较,包括静态/动态场景、室内/室外场景和日间/夜间场景等不同挑战的样例。实验结果证明了我们方法的有效性和稳健性。