In this paper, we propose a multi-objective camera ISP framework that utilizes Deep Reinforcement Learning (DRL) and camera ISP toolbox that consist of network-based and conventional ISP tools. The proposed DRL-based camera ISP framework iteratively selects a proper tool from the toolbox and applies it to the image to maximize a given vision task-specific reward function. For this purpose, we implement total 51 ISP tools that include exposure correction, color-and-tone correction, white balance, sharpening, denoising, and the others. We also propose an efficient DRL network architecture that can extract the various aspects of an image and make a rigid mapping relationship between images and a large number of actions. Our proposed DRL-based ISP framework effectively improves the image quality according to each vision task such as RAW-to-RGB image restoration, 2D object detection, and monocular depth estimation.
翻译:在本文中,我们提出一个多目标相机 ISP 框架,利用深强化学习(DRL)和相机 ISP 工具箱,由基于网络的和传统的 ISP 工具组成。基于 DRL 的相机 ISP 框架从工具箱中迭接地选择一个适当的工具,并将其应用到图像上,以最大限度地发挥特定愿景任务的具体奖赏功能。为此,我们总共实施了51个 ISP 工具,其中包括暴露校正、彩色和色校正、白平衡、精亮、脱色等。我们还提出了高效的 DRL 网络架构,能够提取图像的各个方面,并在图像和大量行动之间建立僵硬的映射关系。我们基于 DRL 的 ISP 框架根据每个愿景任务,如RAW--RGB 图像恢复、 2D 对象探测和单层深度估计,有效地提高了图像质量。