Restoring and inpainting the visual memories that are present, but often impaired, in old photos remains an intriguing but unsolved research topic. Decades-old photos often suffer from severe and commingled degradation such as cracks, defocus, and color-fading, which are difficult to treat individually and harder to repair when they interact. Deep learning presents a plausible avenue, but the lack of large-scale datasets of old photos makes addressing this restoration task very challenging. Here we present a novel reference-based end-to-end learning framework that is able to both repair and colorize old and degraded pictures. Our proposed framework consists of three modules: a restoration sub-network that conducts restoration from degradations, a similarity sub-network that performs color histogram matching and color transfer, and a colorization subnet that learns to predict the chroma elements of images that have been conditioned on chromatic reference signals. The overall system makes use of color histogram priors from reference images, which greatly reduces the need for large-scale training data. We have also created a first-of-a-kind public dataset of real old photos that are paired with ground truth "pristine" photos that have been that have been manually restored by PhotoShop experts. We conducted extensive experiments on this dataset and synthetic datasets, and found that our method significantly outperforms previous state-of-the-art models using both qualitative comparisons and quantitative measurements.
翻译:在旧照片中,现有但经常损坏的视觉记忆的恢复和涂色工作,在旧照片中,旧照片中仍然是一个令人着迷但尚未解析的研究主题。 十年来的照片往往遭受严重和混合降解,如裂纹、脱焦和彩色淡化等严重和混合降解,很难单独处理,在互动时很难修复。 深层次的学习是一个可行的途径,但缺乏大型的旧照片数据集使得缺少大型的旧照片数据集使得恢复任务非常艰巨。 我们在这里展示了一个新的基于参考的端对端对端的比较学习框架,它既能修复旧的和已退化的图片,又使其色彩化。我们提议的框架由三个模块组成:一个恢复退化后恢复的恢复子网络,一个类似性子网络,在它们互动时很难单独处理,很难单独处理。 深层次的学习是一个途径,但缺乏大型的旧照片数据集。 整个系统利用参考图像的颜色缩影缩影模型,这大大降低了大规模培训数据的需要。 我们还创建了第一个分级的子网络,用来从退化的图像中恢复退化公共数据。