We develop a simple and unified framework for nonlinear variable selection that incorporates model uncertainty and is compatible with a wide range of machine learning models (e.g., tree ensembles, kernel methods and neural network). In particular, for a learned nonlinear model $f(\mathbf{x})$, we consider quantifying the importance of an input variable $\mathbf{x}^j$ using the integrated gradient measure $\psi_j = \Vert \frac{\partial}{\partial \mathbf{x}^j} f(\mathbf{x})\Vert^2_2$. We then (1) provide a principled approach for quantifying variable selection uncertainty by deriving its posterior distribution, and (2) show that the approach is generalizable even to non-differentiable models such as tree ensembles. Rigorous Bayesian nonparametric theorems are derived to guarantee the posterior consistency and asymptotic uncertainty of the proposed approach. Extensive simulation confirms that the proposed algorithm outperforms existing classic and recent variable selection methods.
翻译:我们为非线性变量选择开发了一个简单和统一的框架,其中纳入了模型不确定性,并与一系列广泛的机器学习模型(例如树群、内核方法和神经网络)兼容。特别是,对于一个学习的非线性模型$f(\mathbf{x})美元,我们考虑使用综合梯度测量 $\psi_j =\Vert\freac\f-party\phy{x ⁇ j} f(mathbf{x})\Vert%2_2美元来量化输入变量的重要性。然后,我们(1) 提供一个原则性方法,通过推断其后表分布来量化变量的不确定性,(2) 表明这一方法甚至可以普遍适用于无差别的模型,例如树团。 严格地巴伊斯的非参数性可推断出保证拟议方法的后表一致性和随机性不确定性。 范围宽度模拟确认,拟议的算法将现有典型和变量选择方法排出最近的公式。