Gaussian Mixture Reduction with Composite Transportation Divergence

Gaussian mixture reduction (GMR) is the problem of approximating a high order Gaussian mixture by one with lower order. It is widely used in density estimation, recursive tracking in hidden Markov model, and belief propagation. In this work, we show that the GMR can be formulated as an optimization problem which minimizes the composite transportation divergence (CTD) between two mixtures. The optimization problem can be solved by an easy-to-implement Majorization-Minimization (MM) algorithm. We show that the MM algorithm converges under general conditions. One popular computationally efficient approach for GMR is the clustering based iterative algorithms. However, these algorithms lack a theoretical guarantee whether they converge or attain some optimality targets when they do. We show that existing clustering-based algorithms are special cases of our MM algorithm can their theoretical properties are therefore established. We further show the performance of the clustering-based algorithms can be further improved by choosing various cost function in the CTD. Numerical experiments are conducted to illustrate the effectiveness of our proposed extension.