Gaussian process (GP) regression is a fundamental tool in Bayesian statistics. It is also known as kriging and is the Bayesian counterpart to the frequentist kernel ridge regression. Most of the theoretical work on GP regression has focused on a large-$n$ asymptotics, characterising the behaviour of GP regression as the amount of data increases. Fixed-sample analysis is much more difficult outside of simple cases, such as locations on a regular grid. In this work we perform a fixed-sample analysis that was first studied in the context of approximation theory by Driscoll & Fornberg (2002), called the "flat limit". In flat-limit asymptotics, the goal is to characterise kernel methods as the length-scale of the kernel function tends to infinity, so that kernels appear flat over the range of the data. Surprisingly, this limit is well-defined, and displays interesting behaviour: Driscoll & Fornberg showed that radial basis interpolation converges in the flat limit to polynomial interpolation, if the kernel is Gaussian. Leveraging recent results on the spectral behaviour of kernel matrices in the flat limit, we study the flat limit of Gaussian process regression. Results show that Gaussian process regression tends in the flat limit to (multivariate) polynomial regression, or (polyharmonic) spline regression, depending on the kernel. Importantly, this holds for both the predictive mean and the predictive variance, so that the posterior predictive distributions become equivalent. Our results have practical consequences: for instance, they show that optimal GP predictions in the sense of leave-one-out loss may occur at very large length-scales, which would be invisible to current implementations because of numerical difficulties.
翻译:高尔斯进程( GP) 回归是巴耶斯统计中的一个基本工具 。 它也被称为 kriging, 是巴耶斯的对等方 。 GP 回归的理论工作大多侧重于大- $ 零位混凝土, 显示GP 回归的行为随着数据量的增加而变化。 固定的淡化分析在简单案例之外难度更大, 比如在常规网格上的位置 。 在这项工作中, 我们进行固定的回归分析, 这是在Driscoll & Fornberg (2002年) 的近距离理论中首次研究的 。 平流的正轨的回归是“ 膨胀 极限 ” 。 在平流的回归过程中, 目标是将内核回归法方法定性为高尔夫的长度, 高尔夫离子的内向值变压过程可能变得很模糊。 高地平流的底值是, 我们的平流的平流结果将显示为平坦级的平流结果。