Weakly-Supervised Amodal Instance Segmentation with Compositional Priors

Amodal segmentation in biological vision refers to the perception of the entire object when only a fraction is visible. This ability of seeing through occluders and reasoning about occlusion is innate to biological vision but not adequately modeled in current machine vision approaches. A key challenge is that ground-truth supervisions of amodal object segmentation are inherently difficult to obtain. In this paper, we present a neural network architecture that is capable of amodal perception, when weakly supervised with standard (modal) bounding box annotations. Our model extends compositional convolutional neural networks (CompositionalNets), which have been shown to be robust to partial occlusion by explicitly representing objects as a composition of parts. In particular, we extend CompositionalNets to perform three new vision tasks from bounding box supervision only: 1) Learning compositional shape priors of objects in varying 3D poses from modal bounding box supervision; 2) Predicting instance segmentation by integrating the compositional shape priors into the part-voting mechanism in the CompositionalNets; 3) Predicting amodal completion for both the bounding box and the instance segmentation mask by implementing compositional feature alignment in CompositionalNets. Our extensive experiments show that our proposed model can segment amodal masks robustly, with much improved mask prediction qualities compared to state-of-the-art segmentation approaches.