We present the new software OpDiLib, a universal add-on for classical operator overloading AD tools that enables the automatic differentiation (AD) of OpenMP parallelized code. With it, we establish support for OpenMP features in a reverse mode operator overloading AD tool to an extent that was previously only reported on in source transformation tools. We achieve this with an event-based implementation ansatz that is unprecedented in AD. Combined with modern OpenMP features around OMPT, we demonstrate how it can be used to achieve differentiation without any additional modifications of the source code; neither do we impose a priori restrictions on the data access patterns, which makes OpDiLib highly applicable. For further performance optimizations, restrictions like atomic updates on the adjoint variables can be lifted in a fine-grained manner for any parts of the code. OpDiLib can also be applied in a semi-automatic fashion via a macro interface, which supports compilers that do not implement OMPT. In a detailed performance study, we demonstrate the applicability of OpDiLib for a pure operator overloading approach in a hybrid parallel environment. We quantify the cost of atomic updates on the adjoint vector and showcase the speedup and scaling that can be achieved with the different configurations of OpDiLib in both the forward and the reverse pass.
翻译:我们展示了新的软件 OpDiLib, 这是一种通用的软件 OpDiLib, 用于经典操作员超载 AD 工具, 使得 OpenMP 平行代码的自动区分( AD) 。 有了它, 我们就可以在反向模式操作员超载 AD 工具中建立对 Opim MP 功能的支持, 其程度以前只在源转换工具中报告过。 我们通过一个在 AD 上前所没有的以事件为基础的执行 ansatz 实现这一点。 加上在 OMPT 周围的现代 OpenMP 功能, 我们展示了如何在不进一步修改源代码的情况下, 实现差异化; 我们也没有先验地限制数据访问模式, 使得 OpdiLib 高度适用 OpdiLib 。 对于进一步的性能优化, 可以对代码中任何部分的原子变量更新进行细微调整。 OpdiLb 也可以通过宏观界面应用半自动方式应用, 支持不执行 OMPT 。 在详细的业绩研究中, 我们展示了ODIL 和前向式更新的版本格式中, 。