Gaussian Mixture Reduction with Composite Transportation Divergence

Gaussian mixtures can approximate almost any smooth density function and are used to simplify downstream inference tasks. As such, it is widely used in applications in density estimation, belief propagation, and Bayesian filtering. In these applications, a finite Gaussian mixture provides an initial approximation to density functions that are updated recursively. A challenge in these recursions is that the order of the Gaussian mixture increases exponentially, and the inference quickly becomes intractable. To overcome the difficulty, the Gaussian mixture reduction, which approximates a high order Gaussian mixture by one with a lower order, can be used. Existing methods such as the clustering-based approaches are renowned for their satisfactory performance and computationally efficiency. However, they have unknown convergence and optimal targets. We propose a novel optimization-based Gaussian mixture reduction method. We develop a majorization-minimization algorithm for its numerical computation and establish its theoretical convergence under general conditions. We show many existing clustering-based methods are special cases of ours, thus bridging the gap between optimization-based and clustering-based methods. The unified framework allows users to choose the most suitable cost function to achieve superior performance in their specific application. We demonstrate the efficiency and effectiveness of the proposed method through extensive empirical experiments.