Online Multi-Object Tracking and Segmentation with GMPHD Filter and Simple Affinity Fusion

In this paper, we propose a highly practical fully online multi-object tracking and segmentation (MOTS) method that uses instance segmentation results as an input in video. The proposed method exploits the Gaussian mixture probability hypothesis density (GMPHD) filter for online approach which is extended with a hierarchical data association (HDA) and a simple affinity fusion (SAF) model. HDA consists of segment-to-track and track-to-track associations. To build the SAF model, an affinity is computed by using the GMPHD filter that is represented by the Gaussian mixture models with position and motion mean vectors, and another affinity for appearance is computed by using the responses from single object tracker such as the kernalized correlation filters. These two affinities are simply fused by using a score-level fusion method such as Min-max normalization. In addition, to reduce false positive segments, we adopt Mask IoU based merging. In experiments, those key modules, i.e., HDA, SAF, and Mask merging show incremental improvements. For instance, ID-switch decreases by half compared to baseline method. In conclusion, our tracker achieves state-of-the-art level MOTS performance.