Imperative programming allows users to implement their deep neural networks (DNNs) easily and has become an essential part of recent deep learning (DL) frameworks. Recently, several systems have been proposed to combine the usability of imperative programming with the optimized performance of symbolic graph execution. Such systems convert imperative Python DL programs to optimized symbolic graphs and execute them. However, they cannot fully support the usability of imperative programming. For example, if an imperative DL program contains a Python feature with no corresponding symbolic representation (e.g., third-party library calls or unsupported dynamic control flows) they fail to execute the program. To overcome this limitation, we propose Terra, an imperative-symbolic co-execution system that can handle any imperative DL programs while achieving the optimized performance of symbolic graph execution. To achieve this, Terra builds a symbolic graph by decoupling DL operations from Python features. Then, Terra conducts the imperative execution to support all Python features, while delegating the decoupled operations to the symbolic execution. We evaluated the performance improvement and coverage of Terra with ten imperative DL programs for several DNN architectures. The results show that Terra can speed up the execution of all ten imperative DL programs, whereas AutoGraph, one of the state-of-the-art systems, fails to execute five of them.
翻译:硬性编程使用户能够轻松地实施其深层神经网络(DNNS),并成为最近深层学习(DL)框架的重要部分。 最近,一些系统被提议将迫切性编程的可用性与象征性图形执行的优化性能相结合。 这些系统转换了必需要的 Python DL 程序, 以优化符号图形的优化性能并加以执行。 但是, 它们无法充分支持迫切性编程的可用性。 例如, 如果一个必需要的 DL 程序包含一个Python 功能, 但没有相应的象征性代表( 例如第三方图书馆电话或无支持的动态控制流), 它们无法执行程序。 为了克服这一限制, 我们提议Terra, 一个必需要的同步性同步性共同执行系统, 可以处理任何紧迫的 DL 程序, 同时实现最优化的象征性的图形执行性能。 为了实现这一目标, Terra 将 D 系统的所有性能改进和D 执行速度覆盖范围, 我们评估了 Terra L 需要的 D 系统 的自动执行结果 。