Derivatives of differential equation solutions are commonly for parameter estimation, fitting neural differential equations, and as model diagnostics. However, with a litany of choices and a Cartesian product of potential methods, it can be difficult for practitioners to understand which method is likely to be the most effective on their particular application. In this manuscript we investigate the performance characteristics of Discrete Local Sensitivity Analysis implemented via Automatic Differentiation (DSAAD) against continuous adjoint sensitivity analysis. Non-stiff and stiff biological and pharmacometric models, including a PDE discretization, are used to quantify the performance of sensitivity analysis methods. Our benchmarks show that on small systems of ODEs (approximately $<100$ parameters+ODEs), forward-mode DSAAD is more efficient than both reverse-mode and continuous forward/adjoint sensitivity analysis. The scalability of continuous adjoint methods is shown to be more efficient than discrete adjoints and forward methods after crossing this size range. These comparative studies demonstrate a trade-off between memory usage and performance in the continuous adjoint methods that should be considered when choosing the technique, while numerically unstable backsolve techniques from the machine learning literature are demonstrated as unsuitable for most scientific models. The performance of adjoint methods is shown to be heavily tied to the reverse-mode AD method, with tape-based AD methods shown to be 2 orders of magnitude slower on nonlinear partial differential equations than static AD techniques. These results also demonstrate the applicability of DSAAD to differential-algebraic equations, delay differential equations, and hybrid differential equation systems, showcasing an ease of implementation advantage for DSAAD approaches.
翻译:差异方程式解决方案的衍生要素通常用于参数估计,安装神经差异方程式,并用作模型诊断。然而,随着一系列选择以及潜在方法的笛卡尔产品,实践者可能难以理解哪种方法对其特定应用可能最为有效。在本稿中,我们调查通过自动差异分析(DSAAD)实施的差异本地感知分析的性能特征,以进行连续连带敏感度分析。使用非硬性和硬性生物和药理模型,包括PDE分解模型,以量化敏感度分析方法的性能。我们的基准显示,在小型的 ODE系统(约 < 100美元参数+ODEs)上,前方-mode DSAAAD可能比反向前方/联动灵敏度分析更有效。 持续连带方法的伸缩性比在此范围之后的离散性连接和前方方法更高效。这些比较研究显示,在选择最小型的对等价分析方法(约 < 100美元参数+ODODAD)中,前方程式的伸缩法显示,从磁性对正反向式的反向分析方法显示,反向的反向分析方法显示,反向的反向的反向分析方法显示,反向的反向性压法方法显示,反向的反向的反向的反向的反向的反向式的反向式的压法方法展示法方法显示。