Sparse linear models are a gold standard tool for interpretable machine learning, a field of emerging importance as predictive models permeate decision-making in many domains. Unfortunately, sparse linear models are far less flexible as functions of their input features than black-box models like deep neural networks. With this capability gap in mind, we study a not-uncommon situation where the input features dichotomize into two groups: explanatory features, which we wish to explain the model's predictions, and contextual features, which we wish to determine the model's explanations. This dichotomy leads us to propose the contextual lasso, a new statistical estimator that fits a sparse linear model whose sparsity pattern and coefficients can vary with the contextual features. The fitting process involves learning a nonparametric map, realized via a deep neural network, from contextual feature vector to sparse coefficient vector. To attain sparse coefficients, we train the network with a novel lasso regularizer in the form of a projection layer that maps the network's output onto the space of $\ell_1$-constrained linear models. Extensive experiments on real and synthetic data suggest that the learned models, which remain highly transparent, can be sparser than the regular lasso without sacrificing the predictive power of a standard deep neural network.
翻译:开阔线性模型是可解释机器学习的金标准工具, 是一个作为预测模型渗透到许多领域决策的新兴重要领域。 不幸的是, 稀少线性模型与其输入功能的功能功能相比, 与深神经网络等深神经网络等黑盒型模型相比, 其细线性模型的功能灵活性要小得多。 我们想研究一种不常见的情况, 输入特征分解成两组: 解释性特征, 我们希望解释模型的预测, 以及背景特征, 我们希望解释模型的解释。 这种二分法导致我们提出一个背景 lasso, 一个新的统计估计模型, 适合一个稀疏的线性模型, 其宽度模式和系数可以随背景特征变化而变化。 合适的过程包括学习一个非对称地图, 通过深神经网络, 从环境特性矢量到稀薄的系数矢量, 。 为了实现稀薄的系数, 我们用一个新型的固定调节器对网络进行训练, 以预测层的形式将网络的输出映射到$\_ $_ $ $ $ $ $ a concontractracted line 线性模型。 在真实和合成模型上进行广泛的实验, lamod ladestromal lamodal lamodal labal dreal dreal lavelive dal sal slevil sutional sutional sutional subil subil sutional subil subil subil sules subil sutional subild sution subil subil subil subild subil subil subil subil subildal subil subal subil subil subil subil subdal sabildal subaldal sub subdal sub subdal subildal sail ladal ladal saildal ladal ladal ladaldaldal ladal sail 。 。 sail