Self-Validated Ensemble Models for Design of Experiments

In this paper we introduce a new model building algorithm called self-validating ensemble modeling or SVEM. The method enables the fitting of validated predictive models to the relatively small data sets typically generated from designed experiments. We focus on prediction which is often the important metric in studies in bio-pharmaceutical industries. In order to fit validated predictive models, SVEM uses a unique weighting scheme applied to the responses and fractional weighted bootstrapping to generate a large ensemble of fitted models. The weighting scheme allows the original data to serve both as a training set and validation set. The method is very general in application and works with most model selection algorithms. Through extensive simulation studies and a case study we demonstrate that SVEM generates models with lower prediction error as compared to more traditional statistical approaches that are based on fitting a single model when the true model has low sparsity and when the number of experimental runs is small.