A Novel and Fully Automated Domain Transformation Scheme for Near Optimal Surrogate Construction

Recent developments in surrogate construction predominantly focused on two strategies to improve surrogate accuracy. Firstly, component-wise domain scaling informed by cross-validation. Secondly, regression to construct response surfaces using additional information in the form of additional function-values sampled from multi-fidelity models and gradients. Component-wise domain scaling reliably improves the surrogate quality at low dimensions but has been shown to suffer from high computational costs for higher dimensional problems. The second strategy, adding gradients to train surrogates, typically results in regression surrogates. Counter-intuitively, these gradient-enhanced regression-based surrogates do not exhibit improved accuracy compared to surrogates only interpolating function values. This study empirically establishes three main findings. Firstly, constructing the surrogate in poorly scaled domains is the predominant cause of deteriorating response surfaces when regressing with additional gradient information. Secondly, surrogate accuracy improves if the surrogates are constructed in a fully transformed domain, by scaling and rotating the original domain, not just simply scaling the domain. The domain transformation scheme should be based on the local curvature of the approximation surface and not its global curvature. Thirdly, the main benefit of gradient information is to efficiently determine the (near) optimal domain in which to construct the surrogate. This study proposes a foundational transformation algorithm that performs near-optimal transformations for lower dimensional problems. The algorithm consistently outperforms cross-validation-based component-wise domain scaling for higher dimensional problems. A carefully selected test problem set that varies between 2 and 16-dimensional problems is used to clearly demonstrate the three main findings of this study.