This paper studies the theoretical underpinnings of machine learning of ergodic It\^o diffusions. The objective is to understand the convergence properties of the invariant statistics when the underlying system of stochastic differential equations (SDEs) is empirically estimated with a supervised regression framework. Using the perturbation theory of ergodic Markov chains and the linear response theory, we deduce a linear dependence of the errors of one-point and two-point invariant statistics on the error in the learning of the drift and diffusion coefficients. More importantly, our study shows that the usual $L^2$-norm characterization of the learning generalization error is insufficient for achieving this linear dependence result. We find that sufficient conditions for such a linear dependence result are through learning algorithms that produce a uniformly Lipschitz and consistent estimator in the hypothesis space that retains certain characteristics of the drift coefficients, such as the usual linear growth condition that guarantees the existence of solutions of the underlying SDEs. We examine these conditions on two well-understood learning algorithms: the kernel-based spectral regression method and the shallow random neural networks with the ReLU activation function.
翻译:本文研究了机器学习 ERgodic Itção 扩散的理论基础。 目的是在通过受监督的回归框架对随机偏差方程( SDEs) 基础系统进行实验性估计时, 了解异差统计的共性特性。 我们使用ERgodic Markov 链条的扰动理论和线性反应理论, 推断一点和两点差异性统计的误差对学习漂移和扩散系数的误差具有线性依赖性。 更重要的是, 我们的研究显示, 通常的 $L%2$的学习通识错误的中枢特性不足以实现线性依赖性结果。 我们发现, 这种线性依赖性依赖性结果的充足条件是通过学习算法产生统一的Lipschitz 和一致的测算法, 假设空间中保留了某些漂移系数的特性, 例如通常的线性增长条件, 保证存在潜在的SDEs 的解决方案。 我们用两种非常深的学习算法来研究这些条件: 以内核光谱回归回归法和以浅色线性神经网络进行激活功能。