Making Images Real Again: A Comprehensive Survey on Deep Image Composition

As a common image editing operation, image composition aims to cut the foreground from one image and paste it on another image, resulting in a composite image. However, there are many issues that could make the composite images unrealistic. These issues can be summarized as the inconsistency between foreground and background, which includes appearance inconsistency (e.g., incompatible illumination), geometry inconsistency (e.g., unreasonable size), and semantic inconsistency (e.g., mismatched semantic context). Previous works divide image composition task into multiple sub-tasks, in which each sub-task targets at one or more issues. Specifically, object placement aims to find reasonable scale, location, and shape for the foreground. Image blending aims to address the unnatural boundary between foreground and background. Image harmonization aims to adjust the illumination statistics of foreground. Shadow generation aims to generate plausible shadow for the foreground. By putting all the abovementioned efforts together, we can acquire realistic composite images. To the best of our knowledge, there is no previous survey on image composition. In this paper, we conduct comprehensive survey over the sub-tasks of image composition. For each sub-task, we summarize the traditional methods, deep learning based methods, datasets and evaluation. We also point out the limitations of existing methods in each sub-task and the problem of the whole image composition task. Datasets and codes for image composition are summarized at https://github.com/bcmi/Awesome-Image-Composition.