Feedforward neural networks (FNNs) can be viewed as non-linear regression models, where covariates enter the model through a combination of weighted summations and non-linear functions. Although these models have some similarities to the models typically used in statistical modelling, the majority of neural network research has been conducted outside of the field of statistics. This has resulted in a lack of statistically-based methodology, and, in particular, there has been little emphasis on model parsimony. Determining the input layer structure is analogous to variable selection, while the structure for the hidden layer relates to model complexity. In practice, neural network model selection is often carried out by comparing models using out-of-sample performance. However, in contrast, the construction of an associated likelihood function opens the door to information-criteria-based variable and architecture selection. A novel model selection method, which performs both input- and hidden-node selection, is proposed using the Bayesian information criterion (BIC) for FNNs. The choice of BIC over out-of-sample performance as the model selection objective function leads to an increased probability of recovering the true model, while parsimoniously achieving favourable out-of-sample performance. Simulation studies are used to evaluate and justify the proposed method, and applications on real data are investigated.
翻译:进取神经网络(FNNS)可被视为非线性回归模型,其中,共变体通过加权总和和非线性功能结合进入模型。虽然这些模型与统计建模中通常使用的模型有些相似之处,但大部分神经网络研究是在统计领域以外进行的。这导致缺乏基于统计的方法,特别是很少强调模型偏重。确定输入层结构与变量选择相似,而隐藏层的结构则与模型复杂程度相关。在实践中,神经网络模型选择通常通过比较模型,使用模版外性能进行。然而,与此相反,构建相关的可能性功能为基于信息的标准变量和结构选择打开了大门。提议采用新的模型选择方法,既进行投入性选择,又进行隐藏节点选择。选择输入层结构的结构类似于变量选择,而隐藏层结构结构结构结构结构的结构则与模型选择复杂程度相关。在实际选择目标功能上,神经网络模型选择往往通过比较模型来进行选择,使用模版性功能进行比较,同时模拟和模拟性数据应用的模拟和模拟性研究具有更大的可能性。