In this paper, we present D2C-SR, a novel framework for the task of image super-resolution(SR). As an ill-posed problem, the key challenge for super-resolution related tasks is there can be multiple predictions for a given low-resolution input. Most classical methods and early deep learning based approaches ignored this fundamental fact and modeled this problem as a deterministic processing which often lead to unsatisfactory results. Inspired by recent works like SRFlow, we tackle this problem in a semi-probabilistic manner and propose a two-stage pipeline: a divergence stage is used to learn the distribution of underlying high-resolution outputs in a discrete form, and a convergence stage is followed to fuse the learned predictions into a final output. More specifically, we propose a tree-based structure deep network, where each branch is designed to learn a possible high-resolution prediction. At the divergence stage, each branch is trained separately to fit ground truth, and a triple loss is used to enforce the outputs from different branches divergent. Subsequently, we add a fuse module to combine the multiple predictions as the outputs from the first stage can be sub-optimal. The fuse module can be trained to converge w.r.t the final high-resolution image in an end-to-end manner. We conduct evaluations on several benchmarks, including a new proposed dataset with 8x upscaling factor. Our experiments demonstrate that D2C-SR can achieve state-of-the-art performance on PSNR and SSIM, with a significantly less computational cost.
翻译:在本文中,我们提出D2C-SR,这是一个用于图像超分辨率任务的新框架。作为一个不恰当的问题,超级分辨率相关任务的关键挑战在于对特定低分辨率投入的多重预测。大多数古典方法和早期深学习方法忽视了这一基本事实,并将这一问题建为通常导致不满意结果的确定性处理模型。在SRFlow等近期工作启发下,我们以半概率方式处理这一问题,并提议一个两阶段管道:使用一个差异阶段来学习以离散形式分配高分辨率基本产出的分布,并跟踪一个趋同阶段,将所学的高分辨率预测整合到最终产出中。更具体地说,我们提议了一个基于树结构的深度网络,每个分支都是为了学习可能的高分辨率预测。在分歧阶段,每个分支分别接受培训,以适应地面真相,并用三重损失来执行不同部门的产出。随后,我们增加了一个整合模块,将多个预测与第一阶段的高分辨率产出相结合,包括高分辨率和高分辨率标准。我们所培训的模块可以与若干项升级到高分辨率模型。