Artificial neural networks (ANNs) have become a very powerful tool in the approximation of high-dimensional functions. Especially, deep ANNs, consisting of a large number of hidden layers, have been very successfully used in a series of practical relevant computational problems involving high-dimensional input data ranging from classification tasks in supervised learning to optimal decision problems in reinforcement learning. There are also a number of mathematical results in the scientific literature which study the approximation capacities of ANNs in the context of high-dimensional target functions. In particular, there are a series of mathematical results in the scientific literature which show that sufficiently deep ANNs have the capacity to overcome the curse of dimensionality in the approximation of certain target function classes in the sense that the number of parameters of the approximating ANNs grows at most polynomially in the dimension $d \in \mathbb{N}$ of the target functions under considerations. In the proofs of several of such high-dimensional approximation results it is crucial that the involved ANNs are sufficiently deep and consist a sufficiently large number of hidden layers which grows in the dimension of the considered target functions. It is the topic of this work to look a bit more detailed to the deepness of the involved ANNs in the approximation of high-dimensional target functions. In particular, the main result of this work proves that there exists a concretely specified sequence of functions which can be approximated without the curse of dimensionality by sufficiently deep ANNs but which cannot be approximated without the curse of dimensionality if the involved ANNs are shallow or not deep enough.
翻译:人工神经网络(ANNS)在近似高维功能方面已成为一个非常强大的工具。 特别是,由大量隐藏层组成的深层ANNS在一系列实际相关的计算问题中被非常成功地使用,这些问题涉及从监督学习的分类任务到强化学习的最佳决策问题等一系列高维输入数据。 科学文献中也有一些数学结果,这些科学结果研究在高维目标功能方面,ANNS的近似能力。 特别是,在科学文献中的一系列数学结果表明,足够深深的ANNS有能力克服某些目标功能类别近似的深度诅咒,因为高维输入的ANNNS的参数数量在最多的层面增长,在考虑的目标函数的层面是$d\in\mathbb{N}$。 某些高维近维结果的证据表明,所涉及的ANNNNS具有足够深的深度,但是有足够多的隐藏层,在目标函数的深度上没有增加的深度的深度。 这一点由NNF的精度证明,这个核心的精度不能成为这个核心的精度。