Backpropagation (BP) is a core component of the contemporary deep learning incarnation of neural networks. Briefly, BP is an algorithm that exploits the computational architecture of neural networks to efficiently evaluate the gradient of a cost function during neural network parameter optimization. The validity of BP rests on the application of a multivariate chain rule to the computational architecture of neural networks and their associated objective functions. Introductions to deep learning theory commonly present the computational architecture of neural networks in matrix form, but eschew a parallel formulation and justification of BP in the framework of matrix differential calculus. This entails several drawbacks for the theory and didactics of deep learning. In this work, we overcome these limitations by providing a full induction proof of the BP algorithm in matrix notation. Specifically, we situate the BP algorithm in the framework of matrix differential calculus, encompass affine-linear potential functions, prove the validity of the BP algorithm in inductive form, and exemplify the implementation of the matrix form BP algorithm in computer code.
翻译:BP是一种算法,它利用神经网络的计算结构来有效地评价神经网络参数优化期间成本函数的梯度。 BP的有效性取决于对神经网络的计算结构及其相关目标功能应用多变量链规则。引入深层次学习理论通常以矩阵形式呈现神经网络的计算结构,但避免在矩阵差异计算法框架内平行地拟订和解释BP。这包含对深层次学习理论和实践的若干缺陷。在这项工作中,我们克服了这些局限性,在矩阵标记中充分提供了BP算法的上岗证明。具体地说,我们将BP算法置于矩阵差异计算法的框架内,包括直线潜在功能,证明BP算法在导出法中的有效性,并在计算机代码中示范了BP矩阵算法的实施。