In this paper, we study the statistical limits in terms of Sobolev norms of gradient descent for solving inverse problem from randomly sampled noisy observations using a general class of objective functions. Our class of objective functions includes Sobolev training for kernel regression, Deep Ritz Methods (DRM), and Physics Informed Neural Networks (PINN) for solving elliptic partial differential equations (PDEs) as special cases. We consider a potentially infinite-dimensional parameterization of our model using a suitable Reproducing Kernel Hilbert Space and a continuous parameterization of problem hardness through the definition of kernel integral operators. We prove that gradient descent over this objective function can also achieve statistical optimality and the optimal number of passes over the data increases with sample size. Based on our theory, we explain an implicit acceleration of using a Sobolev norm as the objective function for training, inferring that the optimal number of epochs of DRM becomes larger than the number of PINN when both the data size and the hardness of tasks increase, although both DRM and PINN can achieve statistical optimality.
翻译:在本文中,我们研究了从Sobolev梯度下降的规范的统计局限性,以便利用一般的客观功能类别,从随机抽样的噪音观测中解决反向问题。我们的目标功能包括Sobolev对内核回归、深Ritz方法(DRM)和物理知情神经网络(PINN)的培训,以解决作为特殊情况的椭圆偏差方程(PDEs),我们认为,在数据规模和任务强度都增加的情况下,我们模型可能具有无限的参数化作用。我们证明,这个目标功能上的梯度下降也可以实现统计的最佳性以及数据通过抽样规模增加的最佳传递次数。我们根据我们的理论,我们解释说,使用Sobolev规范作为培训的客观功能,意味着在数据规模和任务强度增加的情况下,DRM和PINN都可实现统计的最优化,但DRM和PINN都比PINN多。