Under-display cameras have been proposed in recent years as a way to reduce the form factor of mobile devices while maximizing the screen area. Unfortunately, placing the camera behind the screen results in significant image distortions, including loss of contrast, blur, noise, color shift, scattering artifacts, and reduced light sensitivity. In this paper, we propose an image-restoration pipeline that is ISP-agnostic, i.e. it can be combined with any legacy ISP to produce a final image that matches the appearance of regular cameras using the same ISP. This is achieved with a deep learning approach that performs a RAW-to-RAW image restoration. To obtain large quantities of real under-display camera training data with sufficient contrast and scene diversity, we furthermore develop a data capture method utilizing an HDR monitor, as well as a data augmentation method to generate suitable HDR content. The monitor data is supplemented with real-world data that has less scene diversity but allows us to achieve fine detail recovery without being limited by the monitor resolution. Together, this approach successfully restores color and contrast as well as image detail.
翻译:近些年来,有人提议低射摄像头作为减少移动设备形式因素的方法,同时尽量扩大屏幕区域。 不幸的是,将相机放在屏幕后面,结果造成大量图像扭曲,包括失去对比、模糊、噪音、颜色变化、散射文物和降低光度。 在本文中,我们提议了一种图像恢复管道,即ISP-不可知性,即它可以与任何遗留的ISP相结合,制作一个与使用同一ISP的普通相机外观相匹配的最后图像。这是通过一种深层次的学习方法实现的,即进行RAW到RAW图像的恢复。为了获得大量具有充分对比和场景多样性的真正的低射照相机培训数据,我们进一步开发了一种数据采集方法,利用《人类发展报告》监测器,以及一种数据增强方法来生成适当的《人类发展报告》内容。监测数据得到真实世界数据的补充,而其现场多样性较少,但使我们能够在不受监测分辨率限制的情况下实现细细的恢复。同时,这一方法成功地恢复了颜色和对比以及图像细节细节。