We study two-layer neural networks whose domain and range are Banach spaces with separable preduals. In addition, we assume that the image space is equipped with a partial order, i.e. it is a Riesz space. As the nonlinearity we choose the lattice operation of taking the positive part; in case of $\mathbb R^d$-valued neural networks this corresponds to the ReLU activation function. We prove inverse and direct approximation theorems with Monte-Carlo rates, extending existing results for the finite-dimensional case. In the second part of the paper, we consider training such networks using a finite amount of noisy observations from the regularisation theory viewpoint. We discuss regularity conditions known as source conditions and obtain convergence rates in a Bregman distance in the regime when both the noise level goes to zero and the number of samples goes to infinity at appropriate rates.
翻译:我们研究的是两层神经网络,其域和范围是Banach空间,具有分解的前两个部分。此外,我们假设图像空间配有部分顺序,即Riesz空间。作为非线性,我们选择了以正部分为主的固定操作;如果是$mathbb R ⁇ d$价值的神经网络,则与RELU的激活功能相对应。我们证明,以蒙特-Carlo的速率来反向和直接近比方理论,扩大了现有定数情况的结果。在论文第二部分,我们考虑从定时理论观点的角度使用有限的噪音观察来训练这些网络。我们讨论被称为源条件的常规性条件,并在Rregman的距离内取得比列格曼的趋同率,当噪音水平达到零和样品数量以适当速度走向不精确时。