Under-Display Camera (UDC) has been widely exploited to help smartphones realize full screen display. However, as the screen could inevitably affect the light propagation process, the images captured by the UDC system usually contain flare, haze, blur, and noise. Particularly, flare and blur in UDC images could severely deteriorate the user experience in high dynamic range (HDR) scenes. In this paper, we propose a new deep model, namely UDC-UNet, to address the UDC image restoration problem with the known Point Spread Function (PSF) in HDR scenes. On the premise that Point Spread Function (PSF) of the UDC system is known, we treat UDC image restoration as a non-blind image restoration problem and propose a novel learning-based approach. Our network consists of three parts, including a U-shape base network to utilize multi-scale information, a condition branch to perform spatially variant modulation, and a kernel branch to provide the prior knowledge of the given PSF. According to the characteristics of HDR data, we additionally design a tone mapping loss to stabilize network optimization and achieve better visual quality. Experimental results show that the proposed UDC-UNet outperforms the state-of-the-art methods in quantitative and qualitative comparisons. Our approach won the second place in the UDC image restoration track of MIPI challenge. Codes will be publicly available.
翻译:由于屏幕可能不可避免地影响光传播过程,UDC系统所拍摄的图像通常含有照明、烟雾、模糊和噪音。特别是UDC图像中的耀斑和模糊可能严重恶化在高动态范围(HDR)场景中的用户经验。在本文件中,我们提出了一个新的深层次模型,即UDC-UNet,以解决UDC图像恢复问题,在《人类发展报告》场景中的已知点传播功能(PSF)中解决UDC图像恢复问题。鉴于UDC系统的点传播功能(PSF)已经为人所知,我们把UDC图像恢复视为非盲图像的恢复问题,并提出一种新的基于学习的方法。我们的网络由三个部分组成,包括一个利用多规模信息的Ushape基础网络,一个进行空间变异调的条件分支,以及一个提供给PSF的先前知识的内核分支。根据《人类发展报告》数据的特点,我们将为稳定网络优化和实现更好视觉质量的图像修复方法设计一个音调损失图象。实验结果显示,Ushape-SDSAFS-S-AFS-Salstromagyalst the State State State State State State State State State State Statal Reformlation the Syal Reformal Reformlations