Merging the two cultures of deep and statistical learning provides insights into structured high-dimensional data. Traditional statistical modeling is still a dominant strategy for structured tabular data. Deep learning can be viewed through the lens of generalized linear models (GLMs) with composite link functions. Sufficient dimensionality reduction (SDR) and sparsity performs nonlinear feature engineering. We show that prediction, interpolation and uncertainty quantification can be achieved using probabilistic methods at the output layer of the model. Thus a general framework for machine learning arises that first generates nonlinear features (a.k.a factors) via sparse regularization and stochastic gradient optimisation and second uses a stochastic output layer for predictive uncertainty. Rather than using shallow additive architectures as in many statistical models, deep learning uses layers of semi affine input transformations to provide a predictive rule. Applying these layers of transformations leads to a set of attributes (a.k.a features) to which predictive statistical methods can be applied. Thus we achieve the best of both worlds: scalability and fast predictive rule construction together with uncertainty quantification. Sparse regularisation with un-supervised or supervised learning finds the features. We clarify the duality between shallow and wide models such as PCA, PPR, RRR and deep but skinny architectures such as autoencoders, MLPs, CNN, and LSTM. The connection with data transformations is of practical importance for finding good network architectures. By incorporating probabilistic components at the output level we allow for predictive uncertainty. For interpolation we use deep Gaussian process and ReLU trees for classification. We provide applications to regression, classification and interpolation. Finally, we conclude with directions for future research.
翻译:将深层和统计学习的两种文化合并为深层和高层次数据提供洞察力。传统的统计模型仍然是结构化表层数据的主要战略。深层学习可以通过具有复合链接功能的通用线性模型(GLMs)的透镜来观察。足够的维度减少(SDR)和宽度具有非线性特征工程。我们表明,利用模型输出层的概率方法可以实现预测、内插和不确定性量化。因此,一个机器学习的一般框架首先通过结构化和结构化梯度优化生成非线性特征(a.k.a.a因素),而第二个则利用透析性线性输出层输出层(GLM)来观察预测性不确定性。我们通过精确和快速预测性梯度优化化的输出层(S)来了解深层次的输出结构。我们最终通过透析、透析性化的双轨化(Oralisilal)来研究,通过透析、透析性化的双轨化(Ormalal)来分析结果。