In this paper, we study a real-world JPEG image restoration problem with bit errors on the encrypted bitstream. The bit errors bring unpredictable color casts and block shifts on decoded image contents, which cannot be resolved by existing image restoration methods mainly relying on pre-defined degradation models in the pixel domain. To address these challenges, we propose a robust JPEG decoder, followed by a two-stage compensation and alignment framework to restore bitstream-corrupted JPEG images. Specifically, the robust JPEG decoder adopts an error-resilient mechanism to decode the corrupted JPEG bitstream. The two-stage framework is composed of the self-compensation and alignment (SCA) stage and the guided-compensation and alignment (GCA) stage. The SCA adaptively performs block-wise image color compensation and alignment based on the estimated color and block offsets via image content similarity. The GCA leverages the extracted low-resolution thumbnail from the JPEG header to guide full-resolution pixel-wise image restoration in a coarse-to-fine manner. It is achieved by a coarse-guided pix2pix network and a refine-guided bi-directional Laplacian pyramid fusion network. We conduct experiments on three benchmarks with varying degrees of bit error rates. Experimental results and ablation studies demonstrate the superiority of our proposed method. The code will be released at https://github.com/wenyang001/Two-ACIR.
翻译:在本文中,我们研究了一个真实的JPEG图像恢复问题,即加密比特流上的比特错误。比特错误会给解码图像内容带来不可预测的色彩失真和块位移,这不能通过现有的主要依靠像素域内预定义的退化模型的图像恢复方法来解决。为了解决这些挑战,我们提出了一个强健的JPEG解码器,后跟一个二阶段补偿和对齐框架来恢复比特流损坏的JPEG图像。具体来说,强健的JPEG解码器采用一种错误鲁棒机制来解码损坏的JPEG比特流。两阶段框架由自补偿和对齐(SCA)阶段和导向补偿和对齐(GCA)阶段组成。SCA根据通过图像内容相似性估计的颜色和块偏移自适应地执行块状图像颜色补偿和对齐。GCA利用从JPEG标头中提取的低分辨率缩略图以粗略到精细的方式指导全分辨率像素级图像恢复。它由一个粗略引导pix2pix网络和一个细分引导的双向拉普拉斯金字塔融合网络实现。我们在三个基准测试中进行了实验,比特错误率不同。实验结果和消融研究证明了我们提出的方法的优越性。代码将在https://github.com/wenyang001/Two-ACIR发布。