We present PNKH-B, a projected Newton-Krylov method for iteratively solving large-scale optimization problems with bound constraints. PNKH-B is geared toward situations in which function and gradient evaluations are expensive, and the (approximate) Hessian is only available through matrix-vector products. This is commonly the case in large-scale parameter estimation, machine learning, and image processing. In each iteration, PNKH-B uses a low-rank approximation of the (approximate) Hessian to determine the search direction and construct the metric used in a projected line search. The key feature of the metric is its consistency with the low-rank approximation of the Hessian on the Krylov subspace. This renders PNKH-B similar to a projected variable metric method. We present an interior point method to solve the quadratic projection problem efficiently. Since the interior point method effectively exploits the low-rank structure, its computational cost only scales linearly with respect to the number of variables, and it only adds negligible computational time. We also experiment with variants of PNKH-B that incorporate estimates of the active set into the Hessian approximation. We prove the global convergence to a stationary point under standard assumptions. Using three numerical experiments motivated by parameter estimation, machine learning, and image reconstruction, we show that the consistent use of the Hessian metric in PNKH-B leads to fast convergence, particularly in the first few iterations. We provide our MATLAB implementation at https://github.com/EmoryMLIP/PNKH-B.
翻译:我们展示了预测的牛顿-克利洛夫(PNKH-B)方法,用于在约束性限制下迭接地解决大规模优化问题。 PNKH-B(PNKH-B)是针对功能和梯度评价费用昂贵和(近似)Hessian(Hissian)只能通过矩阵-矢量产品获得。在大规模参数估计、机器学习和图像处理中,通常都是这种情况。在每次迭代中,PNKH-B(近似)使用低端近似(低端 ) Hession( Hesusian) 来确定搜索方向和构建预测线搜索中使用的衡量标准。该指标的关键特征是它与Krylov 子空间的赫森(Hissian) 低端近端近似于成本和梯度评价,使PNKH- AT- B(近端) 与预测的可变度测量方法相似。我们用内部点方法有效地利用了低端结构,其计算成本仅为线性第一比值,而只增加微不足道的计算时间。我们还在HNKH-B(H-B) 模型的变换变式,我们用了3个不断的MLILILA(我们使用的MLIL) 的模型的模型的模型的模型的模型的模型的模拟,我们用了连续的模型的模型的模型的模型的模型的模型的精确度的模拟的模拟的模型的模拟的模拟的精确度估算。