In this article, we propose and develop a novel Bayesian algorithm for optimization of functions whose first and second partial derivatives are known. The basic premise is the Gaussian process representation of the function which induces a first derivative process that is also Gaussian. The Bayesian posterior solutions of the derivative process set equal to zero, given data consisting of suitable choices of input points in the function domain and their function values, emulate the stationary points of the function, which can be fine-tuned by setting restrictions on the prior in terms of the first and second derivatives of the objective function. These observations motivate us to propose a general and effective algorithm for function optimization that attempts to get closer to the true optima adaptively with in-built iterative stages. We provide theoretical foundation to this algorithm, proving almost sure convergence to the true optima as the number of iterative stages tends to infinity. The theoretical foundation hinges upon our proofs of almost sure uniform convergence of the posteriors associated with Gaussian and Gaussian derivative processes to the underlying function and its derivatives in appropriate fixed-domain infill asymptotics setups; rates of convergence are also available. We also provide Bayesian characterization of the number of optima using information inherent in our optimization algorithm. We illustrate our Bayesian optimization algorithm with five different examples involving maxima, minima, saddle points and even inconclusiveness. Our examples range from simple, one-dimensional problems to challenging 50 and 100-dimensional problems.
翻译:在本篇文章中,我们提出并发展了一种新型的贝叶西亚算法,以优化第一个和第二个部分衍生物已知的功能。基本前提是高萨进程代表功能的功能,它引导第一个衍生物进程,同时也是高萨。贝叶西亚衍生工艺的后继解决方案设定为零,数据包括功能域及其功能值中适当选择的输入点及其功能值,并效仿该功能的固定点,可以通过在目标函数的第一和第二衍生物方面对先前的功能设置限制加以细微调整。这些观察促使我们提出一种通用和有效的功能优化算法,试图通过内部迭代阶段适应性更接近真正的选择。我们为这一算法提供了理论基础,几乎可以证明与真正的选择一致,因为迭代阶段的数量往往不尽相同。理论基础取决于我们证明与高山和高尔西亚衍生物工艺相关的后级几乎一致,与基本功能及其衍生物的匹配性功能及其衍生物。我们用一个固定的固定式算法来填补一个简单的定型结构的定型结构,我们用五种模型来说明我们最迭阶段的定型阶段的定式结构。