Learning the principal eigenfunctions of an integral operator defined by a kernel and a data distribution is at the core of many machine learning problems. Traditional nonparametric solutions based on the Nystr{\"o}m formula suffer from scalability issues. Recent work has resorted to a parametric approach, i.e., training neural networks to approximate the eigenfunctions. However, the existing method relies on an expensive orthogonalization step and is difficult to implement. We show that these problems can be fixed by using a new series of objective functions that generalizes the EigenGame~\citep{gemp2020eigengame} to function space. We test our method on a variety of supervised and unsupervised learning problems and show it provides accurate approximations to the eigenfunctions of polynomial, radial basis, neural network Gaussian process, and neural tangent kernels. Finally, we demonstrate our method can scale up linearised Laplace approximation of deep neural networks to modern image classification datasets through approximating the Gauss-Newton matrix. Code is available at \url{https://github.com/thudzj/neuraleigenfunction}.
翻译:学习由内核和数据分布定义的有机操作者的主要天体功能是许多机器学习问题的核心。基于 Nystr 的公式的传统非参数解决方案存在可缩放问题。 最近的工作采用了参数学方法, 即培训神经网络以近似天体功能。 但是, 现有方法依赖于昂贵的正向分解步骤, 并且难以执行。 我们显示, 这些问题可以通过使用一系列新的客观功能来解决, 将 EigenGame ⁇ citep{ gemp202020egengame} 概括为功能空间。 我们测试了我们的方法, 测试了各种受监督和不受监督的学习问题, 并展示了我们的方法, 提供了与多核、 辐射基础、 神经网络测量过程和 内核内核内核的机能功能的精确近似值。 最后, 我们证明我们的方法可以将深神经网络的直线化近似值提升为现代图像分类数据集, 通过对 Gagormummus/Newmajuralmaus/Newgillism。