This paper develops simple feed-forward neural networks that achieve the universal approximation property for all continuous functions with a fixed finite number of neurons. These neural networks are simple because they are designed with a simple and computable continuous activation function $\sigma$ leveraging a triangular-wave function and a softsign function. We prove that $\sigma$-activated networks with width $36d(2d+1)$ and depth $11$ can approximate any continuous function on a $d$-dimensioanl hypercube within an arbitrarily small error. Hence, for supervised learning and its related regression problems, the hypothesis space generated by these networks with a size not smaller than $36d(2d+1)\times 11$ is dense in the space of continuous functions. Furthermore, classification functions arising from image and signal classification are in the hypothesis space generated by $\sigma$-activated networks with width $36d(2d+1)$ and depth $12$, when there exist pairwise disjoint closed bounded subsets of $\mathbb{R}^d$ such that the samples of the same class are located in the same subset.
翻译:本文开发了简单的向导神经网络, 以固定数量神经元实现所有连续功能的通用近似属性。 这些神经网络很简单, 因为设计时具有简单和可计算的持续激活功能$\sigma$, 以三角波函数和软信号函数为杠杆。 我们证明, $\sgma$ 激活网络, 宽度为 36d(2d+1) 美元, 深度为 111 美元, 可以在任意的小错误中, 任何美元- dimensioanel 超立方的连续功能上达到 。 因此, 为了有监督的学习及其相关的回归问题, 这些网络产生的假设空间不小于 36d( 2d+1)\ 乘以连续功能空间的面积为 11 。 此外, 图像和信号分类产生的分类功能位于假设空间, 宽度为 36d(2d+1) 美元和深度为1美元, 深度为1美元, 任何连续的功能都相当于 $\\\\\ mathb{R\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\