Stochastic scientific models and machine learning optimization estimators have a large number of variables; hence computing large sparse Jacobians and Hessians is important. Algorithmic differentiation (AD) greatly reduces the programming effort required to obtain the sparsity patterns and values for these matrices. We present forward, reverse, and subgraph methods for computing sparse Jacobians and Hessians. Special attention is given the the subgraph method because it is new. The coloring and compression steps are not necessary when computing sparse Jacobians and Hessians using subgraphs. Complexity analysis shows that for some problems the subgraph method is expected to be much faster. We compare C++ operator overloading implementations of the methods in the ADOL-C and CppAD software packages using some of the MINPACK-2 test problems. The experiments are set up in a way that makes them easy to run on different hardware, different systems, different compilers, other test problem and other AD packages. The setup time is the time to record the graph, compute sparsity, coloring, compression, and optimization of the graph. If the setup is necessary for each evaluation, the subgraph implementation has similar run times for sparse Jacobians and faster run times for sparse Hessians.
翻译:存储科学模型和机器学习优化估计器有许多变量; 因此计算大量稀少的雅各布人和黑森人是重要的。 算法差异化( AD) 极大地减少了获取这些矩阵的宽度模式和值所需的程序设计工作。 我们为计算稀少的雅各布人和黑森人提供了前方、 反向和子图方法。 特别注意到子图方法是新的。 使用子图计算稀少的雅各布人和黑森人时, 色化和压缩步骤是不必要的。 复杂度分析表明, 对于某些问题, 子图方法预计会更快。 我们比较了使用某些 MINPACK-2 测试问题的AD- C 和 CppAD软件包实施超载方法的C++ 操作员。 实验设置的方式使他们更容易运行不同的硬件、 不同的系统、 不同的汇编器、 其他测试问题和其他 AD 软件包。 设置时间是记录图表、 计算孔径、 颜色、 压缩、 压缩和 优化图形 的时间。 如果他设置的系统在每次执行时都比亚时要更快。 。, 设置, 需要 进行 快速 。