Feature Selection (FS), such as filter, wrapper, and embedded methods, aims to find the optimal feature subset for a given downstream task. However, in many real-world practices, 1) the criteria of FS vary across domains; 2) FS is brittle when data is a high-dimensional and small sample size. Can selected feature subsets be more generalized, accurate, and input dimensionality agnostic? We generalize this problem into a deep differentiable feature selection task and propose a new perspective: discrete feature subsetting as continuous embedding space optimization. We develop a generic and principled framework including a deep feature subset encoder, accuracy evaluator, decoder, and gradient ascent optimizer. This framework implements four steps: 1) features-accuracy training data preparation; 2) deep feature subset embedding; 3) gradient-optimized search; 4) feature subset reconstruction. We develop new technical insights: reinforcement as a training data generator, ensembles of diverse peer and exploratory feature selector knowledge for generalization, an effective embedding from feature subsets to continuous space along with joint optimizing reconstruction and accuracy losses to select accurate features. Experimental results demonstrate the effectiveness of the proposed method.
翻译:筛选、 包装和嵌入方法等特性选择 (FS) 旨在为特定下游任务找到最佳特性子集。 但是,在许多现实世界做法中,1 FS的标准在不同领域之间有差异;2 FS在数据具有高维度和小样本大小时是困难的。 选定特性子集能够更加普及、准确和输入维度是不可知性的吗? 我们将这一问题概括为深层次的特性选择任务,并提出新的视角: 离散特性子集成作为连续嵌入空间优化。 我们开发了一个通用和原则性框架,包括一个深特性子集、准确性评估器、解码器和梯度作为精度优化器。 这个框架实施四个步骤:(1) 特征精确性培训数据编制;(2) 深度特性子集嵌入;(3) 梯度优化搜索;(4) 特征子集重建。 我们开发新的技术洞察力: 强化作为培训数据生成器, 组合各种同级和探索性特征选样知识, 有效地从特性子集入持续空间,同时联合优化重建和准确性损失以选择精确性特征特征特征特征特征特征。</s>