ML-KFHE: Multi-label ensemble classification algorithm exploiting sensor fusion properties of the Kalman filter

Despite the success of ensemble classification methods in multi-class classification problems, ensemble methods based on approaches other than bagging have not been widely explored for multi-label classification problems. The Kalman Filter-based Heuristic Ensemble (KFHE) is a recent ensemble method that exploits the sensor fusion properties of the Kalman filter to combine several classifier models, and that has been shown to be very effective. This work proposes a multi-label version of KFHE, ML-KFHE, demonstrating the effectiveness of the KFHE method on multi-label datasets. Two variants are introduced based on the underlying component classifier algorithm, ML-KFHE-HOMER, and ML-KFHE-CC which uses HOMER and Classifier Chain (CC) as the underlying multi-label algorithms respectively. ML-KFHE-HOMER and ML-KFHE-CC sequentially trains multiple HOMER and CC multi-label classifiers and aggregates their outputs using the sensor fusion properties of the Kalman filter. Experiments and detailed analysis performed on thirteen multi-label datasets and eight other algorithms, including state-of-the-art ensemble methods, show that for both versions, the ML-KFHE framework improves the ensembling process significantly with respect to bagging based combinations of HOMER and CC, thus demonstrating the effectiveness of ML-KFHE. Also, the ML-KFHE-HOMER variant was found to perform consistently and significantly better than existing multi-label methods including existing approaches based on ensembles.