Using tools from topology and functional analysis, we provide a framework where artificial neural networks, and their architectures, can be formally described. We define the notion of machine in a general topological context and show how simple machines can be combined into more complex ones. We explore finite- and infinite-depth machines, which generalize neural networks and neural ordinary differential equations. Borrowing ideas from functional analysis and kernel methods, we build complete, normed, infinite-dimensional spaces of machines, and we discuss how to find optimal architectures and parameters -- within those spaces -- to solve a given computational problem. In our numerical experiments, these kernel-inspired networks can outperform classical neural networks when the training dataset is small.
翻译:使用来自地形学和功能分析的工具,我们提供了一个框架,可以正式描述人造神经网络及其结构。我们从一般的地形学角度界定机器的概念,并展示如何将简单的机器结合到更复杂的结构中。我们探索了有限和无限的深度机器,这些机器将神经网络和普通神经差异方程式普遍化。我们从功能分析和内核方法中借取思想,我们建立完整、规范、无限的机器空间,我们讨论如何在这些空间中找到最佳的建筑和参数,以解决一个特定的计算问题。在我们的数字实验中,这些内核激发的网络可以在培训数据集小的时候超越古典神经网络。