In this paper, we develop a modified differential Structure from Motion (SfM) algorithm that can estimate relative pose from two consecutive frames despite of Rolling Shutter (RS) artifacts. In particular, we show that under constant velocity assumption, the errors induced by the rolling shutter effect can be easily rectified by a linear scaling operation on each optical flow. We further propose a 9-point algorithm to recover the relative pose of a rolling shutter camera that undergoes constant acceleration motion. We demonstrate that the dense depth maps recovered from the relative pose of the RS camera can be used in a RS-aware warping for image rectification to recover high-quality Global Shutter (GS) images. Experiments on both synthetic and real RS images show that our RS-aware differential SfM algorithm produces more accurate results on relative pose estimation and 3D reconstruction from images distorted by RS effect compared to standard SfM algorithms that assume a GS camera model. We also demonstrate that our RS-aware warping for image rectification method outperforms state-of-the-art commercial software products, i.e. Adobe After Effects and Apple Imovie, at removing RS artifacts.
翻译:在本文中,我们开发了来自运动(SfM)算法的改良差分结构,该算法可以估计两个连续框架的相对面貌,尽管有滚动开关(RS)的人工制品。特别是,我们显示,在持续速度假设下,滚动开关效应引起的错误很容易通过对每个光学流的线性缩放操作得到纠正。我们进一步提出一个9点算法,以恢复一个不断加速动作的滚动开关相机的相对面貌。我们证明,从RS相机相对面部中回收的密度深度图可以用于一个对图像进行有意识的转换,以恢复高质量的全球开关器(GS)图像。合成和真实的RS图像实验显示,我们的RS-觉差分S的SfM算法在相对面估计和3D的重建方面产生更准确的结果,而由RSS效应扭曲的SfM算法则假定一个GS摄像模型。我们还表明,我们的RS-觉对图像校正方法的刻度图,可以用于对立的状态-艺术商业软件产品进行校正,即AAdebe Feffy 后和Amas Imoview,在RS。