Two-stage Design for Failure Probability Estimation with Gaussian Process Surrogates

We tackle the problem of quantifying failure probabilities for expensive deterministic computer experiments with stochastic inputs. The computational cost of the computer simulation prohibits direct Monte Carlo (MC) and necessitates a surrogate model, turning the problem into a two-stage enterprise (surrogate training followed by probability estimation). Limited budgets create a design problem: how should expensive evaluations be allocated between and within the training and estimation stages? One may use the entire evaluation budget to sequentially train the surrogate through contour location (CL), with failure probabilities then estimated solely from the surrogate (we call it "surrogate MC"). But extended CL offers diminishing returns, and surrogate MC relies too stringently on surrogate accuracy. Alternatively, a partially trained surrogate may inform importance sampling, but this can provide erroneous results when budgets are limited. Instead we propose a two-stage design: starting with sequential CL, halting CL once learning has plateaued, then greedily allocating the remaining budget to MC samples with high classification entropy. Ultimately, we employ a "hybrid MC" estimator which leverages the trained surrogate in conjunction with the true responses observed in this second stage. Our unique two-stage design strikes an appropriate balance between exploring and exploiting, and outperforms alternatives, including both of the aforementioned approaches, on a variety of benchmark exercises. With these tools, we are able to effectively estimate small failure probabilities with only hundreds of simulator evaluations, showcasing functionality with both shallow and deep Gaussian process surrogates, and deploying our method on a simulation of fluid flow around an airfoil.