Machine learning and neural network models in particular have been improving the state of the art performance on many artificial intelligence related tasks. Neural network models are typically implemented using frameworks that perform gradient based optimization methods to fit a model to a dataset. These frameworks use a technique of calculating derivatives called automatic differentiation (AD) which removes the burden of performing derivative calculations from the model designer. In this report we describe AD, its motivations, and different implementation approaches. We briefly describe dataflow programming as it relates to AD. Lastly, we present example programs that are implemented with Tensorflow and PyTorch, which are two commonly used AD frameworks.
翻译:特别是机器学习和神经网络模型一直在改善许多人工智能相关任务的先进性能; 神经网络模型通常使用采用基于梯度优化方法的框架来实施,以适应数据集中的模型; 这些框架使用一种计算衍生物的技术,称为自动区分(AD),从模型设计者那里消除进行衍生物计算的负担; 在本报告中,我们描述AD、其动机和不同的执行方法; 我们简单描述与AD有关的数据流编程; 最后,我们介绍与Tensorflow和PyTorrch执行的方案,这是两个常用的AD框架。