This paper presents a novel feature selection method based on the conditional mutual information (CMI). The proposed High Order Conditional Mutual Information Maximization (HOCMIM) incorporates high order dependencies into the feature selection procedure and has a straightforward interpretation due to its bottom-up derivation. The HOCMIM is derived from the CMI's chain expansion and expressed as a maximization optimization problem. The maximization problem is solved using a greedy search procedure, which speeds up the entire feature selection process. The experiments are run on a set of benchmark datasets (20 in total). The HOCMIM is compared with eighteen state-of-the-art feature selection algorithms, from the results of two supervised learning classifiers (Support Vector Machine and K-Nearest Neighbor). The HOCMIM achieves the best results in terms of accuracy and shows to be faster than high order feature selection counterparts.
翻译:本文件介绍了基于有条件的相互信息的新颖特征选择方法(CMI)。拟议的高秩序有条件的相互信息最大化(HOMIM)将高度依赖性纳入特征选择程序,并因其自下而上衍生而直截了当地加以解释。HOMIM源自于CMI的链条扩展,表述为最大化优化问题。最大化问题通过一种贪婪的搜索程序来解决,该程序加快了整个特征选择过程。实验运行在一套基准数据集上(共20个)。HOCIM与两个受监督的学习分类师(支持矢量机器和K-Nearest Neighbor)的18个最先进的特征选择算法相比较。HOMIM在准确性方面取得了最佳效果,并显示比高排序特征选择对应方更快。