Ordering-Based Kalman Filter Selective Ensemble for Classification

This paper investigates Kalman Filter-based Heuristic Ensemble (KFHE), which is a new perspective on multi-class ensemble classification with performance significantly better or at least as good as the state-of-the-art algorithms. We prove that the sample weight tuning method used in KFHE is a version of adaptive boosting, and the weight distribution does not change anymore and leads to redundant classifiers when the algorithm iterates enough times. This motivates us to select a sub-ensemble to alleviate the redundancy and improve the performance of the ensemble. An Ordering-based Kalman Filter Selective Ensemble (OKFSE) is proposed in this paper to select a sub-ensemble using the margin distance minimization approach. We demonstrate the effectiveness and robustness of OKFSE through extensive experiments on 20 real-world UCI datasets, and the statistical test shows that OKFSE significantly outperforms the state-of-the-art KFHE and clustering-based pruning methods on these datasets with 5% and 10% class label noise.