Real engineering and scientific applications often involve one or more qualitative inputs. Standard Gaussian processes (GPs), however, cannot directly accommodate qualitative inputs. The recently introduced latent variable Gaussian process (LVGP) overcomes this issue by first mapping each qualitative factor to underlying latent variables (LVs), and then uses any standard GP covariance function over these LVs. The LVs are estimated similarly to the other GP hyperparameters through maximum likelihood estimation, and then plugged into the prediction expressions. However, this plug-in approach will not account for uncertainty in estimation of the LVs, which can be significant especially with limited training data. In this work, we develop a fully Bayesian approach for the LVGP model and for visualizing the effects of the qualitative inputs via their LVs. We also develop approximations for scaling up LVGPs and fully Bayesian inference for the LVGP hyperparameters. We conduct numerical studies comparing plug-in inference against fully Bayesian inference over a few engineering models and material design applications. In contrast to previous studies on standard GP modeling that have largely concluded that a fully Bayesian treatment offers limited improvements, our results show that for LVGP modeling it offers significant improvements in prediction accuracy and uncertainty quantification over the plug-in approach.
翻译:潜变量高斯过程模型的全贝叶斯推理
真实的工程和科学应用常常涉及一个或多个定性输入。标准的高斯过程 (GP)不能直接容纳定性输入。最近引入的潜变量高斯过程 (LVGP)通过首先将每个定性因素映射到潜在变量 (LVs)上,然后在这些LVs上使用任何标准的GP协方差函数来解决这个问题。LVs通过最大似然估计方式与其他GP超参数一样进行估计,然后插入到预测表达式中。然而,这种插入方法不会考虑LVs估计的不确定性,特别是在训练数据有限的情况下,这种不确定性可能是显著的。在这项工作中,我们为LVGP模型开发了一种全贝叶斯方法,并通过其LVs可视化定性输入的效果。我们还开发了适用于扩大LVGP和全贝叶斯推理LVGP超参数的近似方法。我们通过几个工程模型和材料设计应用的数值研究比较了插入推断和全贝叶斯推断。与以往的标准GP建模研究大多得出结论全贝叶斯处理仅在预测精度上提供有限的改进不同,我们的结果表明,对于LVGP建模,全贝叶斯处理在预测准确性和不确定性量化方面比插入方法提供了显著的改进。