This paper aims at exploring how to synthesize close-to-real blurs that existing video deblurring models trained on them can generalize well to real-world blurry videos. In recent years, deep learning-based approaches have achieved promising success on video deblurring task. However, the models trained on existing synthetic datasets still suffer from generalization problems over real-world blurry scenarios with undesired artifacts. The factors accounting for the failure remain unknown. Therefore, we revisit the classical blur synthesis pipeline and figure out the possible reasons, including shooting parameters, blur formation space, and image signal processor~(ISP). To analyze the effects of these potential factors, we first collect an ultra-high frame-rate (940 FPS) RAW video dataset as the data basis to synthesize various kinds of blurs. Then we propose a novel realistic blur synthesis pipeline termed as RAW-Blur by leveraging blur formation cues. Through numerous experiments, we demonstrate that synthesizing blurs in the RAW space and adopting the same ISP as the real-world testing data can effectively eliminate the negative effects of synthetic data. Furthermore, the shooting parameters of the synthesized blurry video, e.g., exposure time and frame-rate play significant roles in improving the performance of deblurring models. Impressively, the models trained on the blurry data synthesized by the proposed RAW-Blur pipeline can obtain more than 5dB PSNR gain against those trained on the existing synthetic blur datasets. We believe the novel realistic synthesis pipeline and the corresponding RAW video dataset can help the community to easily construct customized blur datasets to improve real-world video deblurring performance largely, instead of laboriously collecting real data pairs.
翻译:本文旨在探索如何综合近到真实的模糊性, 即所培训的现有视频模糊性模型能够将其概括为真实世界的模糊性视频。 近年来, 深层次的基于学习的方法在视频模糊性任务上取得了大有希望的成功。 然而, 现有合成数据集培训的模型仍然在现实世界的模糊性假想中遇到概括性的问题, 并带有不理想的人工制品。 导致失败的因素仍然未知。 因此, 我们重新审视经典的模糊性合成管道, 并找出可能的原因, 包括射击参数、 模糊的形成空间 和图像信号处理 ~( ISP ) 。 为了分析这些潜在因素的影响, 我们首先收集了超高的框架率( 940 FPS), 将视频数据集作为综合各种模糊性能的数据基础。 然后我们提出一个新的现实的模糊性合成管道, 使用模糊的编程信号提示。 我们通过无数的实验, 将精细化的模糊性合成空间的模糊性变色性, 采用相同的 ISP 作为真实性测试数据, 能够有效消除合成数据的负面的内脏性数据效果。