The kernel function and its hyperparameters are the central model selection choice in a Gaussian proces (Rasmussen and Williams, 2006). Typically, the hyperparameters of the kernel are chosen by maximising the marginal likelihood, an approach known as Type-II maximum likelihood (ML-II). However, ML-II does not account for hyperparameter uncertainty, and it is well-known that this can lead to severely biased estimates and an underestimation of predictive uncertainty. While there are several works which employ a fully Bayesian characterisation of GPs, relatively few propose such approaches for the sparse GPs paradigm. In this work we propose an algorithm for sparse Gaussian process regression which leverages MCMC to sample from the hyperparameter posterior within the variational inducing point framework of Titsias (2009). This work is closely related to Hensman et al. (2015b) but side-steps the need to sample the inducing points, thereby significantly improving sampling efficiency in the Gaussian likelihood case. We compare this scheme against natural baselines in literature along with stochastic variational GPs (SVGPs) along with an extensive computational analysis.
翻译:内核函数及其超光度是Gaussian proces(Ramsmussen and Williams, 2006, 2006)中的核心模型选择选择模式选择(Ramsmussen and Williams, 2006, 2006)通常选择内核的超光度计,办法是通过最大限度地提高边际可能性(一种称为Type-II最大可能性(ML-II)的方法)来选择的。然而,ML-II并不考虑超光度不确定性,众所周知,这可能导致严重偏差估计和低估预测不确定性。虽然有几项工作采用全巴伊西亚GPs的特征,但相对较少的为稀薄的GPs模式提出这种方法。在这项工作中,我们提出了稀少高斯进程回归的算法,该算法在Titsias(2009年)变异引点框架内利用MCMMC从超参数后表镜取样。这项工作与Hensman et al. (2015b) 密切相关,但从需要抽样测导点,从而大大提高高斯概率案例的采样效率。我们将这一计划与大量GPAS变异分析同文献中的自然基线进行比较。