反向传播一词严格来说仅指用于计算梯度的算法,而不是指如何使用梯度。但是该术语通常被宽松地指整个学习算法,包括如何使用梯度,例如通过随机梯度下降。反向传播将增量计算概括为增量规则中的增量规则,该规则是反向传播的单层版本,然后通过自动微分进行广义化,其中反向传播是反向累积(或“反向模式”)的特例。 在机器学习中,反向传播(backprop)是一种广泛用于训练前馈神经网络以进行监督学习的算法。对于其他人工神经网络(ANN)都存在反向传播的一般化–一类算法,通常称为“反向传播”。反向传播算法的工作原理是,通过链规则计算损失函数相对于每个权重的梯度,一次计算一层,从最后一层开始向后迭代,以避免链规则中中间项的冗余计算。

VIP内容

图神经网络(GNN)是一类基于深度学习的处理图域信息的方法,它通过将图广播操作和深度学习算法结合,可以让图的结构信息和顶点属性信息都参与到学习中,在顶点分类、图分类、链接预测等应用中表现出良好的效果和可解释性,已成为一种广泛应用的图分析方法.然而现有主流的深度学习框架(如Tensorflow、PyTorch等)没有为图神经网络计算提供高效的存储支持和图上的消息传递支持,这限制了图神经网络算法在大规模图数据上的应用.目前已有诸多工作针对图结构的数据特点和图神经网络的计算特点,探索了大规模图神经网络系统的设计和实现方案.本文首先对图神经网络的发展进行简要概述,总结了设计图神经网络系统需要面对的挑战;随后对目前图神经网络系统的工作进行介绍,从系统架构、编程模型、消息传递优化、图分区策略、通信优化等多个方面对系统进行分析;最后使用部分已开源的图神经网络系统进行实验评估,从精确度、性能、扩展性等多个方面验证这些系统的有效性.

http://www.jos.org.cn/jos/ch/reader/view_abstract.aspx?file_no=6311

成为VIP会员查看完整内容
0
86

最新论文

Quadratic programs (QPs) that enforce control barrier functions (CBFs) have become popular for safety-critical control synthesis, in part due to their ease of implementation and constraint specification. The construction of valid CBFs, however, is not straightforward, and for arbitrarily chosen parameters of the QP, the system trajectories may enter states at which the QP either eventually becomes infeasible, or may not achieve desired performance. In this work, we pose the control synthesis problem as a differential policy whose parameters are optimized for performance over a time horizon at high level, thus resulting in a bi-level optimization routine. In the absence of knowledge of the set of feasible parameters, we develop a Recursive Feasibility Guided Gradient Descent approach for updating the parameters of QP so that the new solution performs at least as well as previous solution. By considering the dynamical system as a directed graph over time, this work presents a novel way of optimizing performance of a QP controller over a time horizon for multiple CBFs by (1) using the gradient of its solution with respect to its parameters by employing sensitivity analysis, and (2) backpropagating these as well as system dynamics gradients to update parameters while maintaining feasibility of QPs.

0
0
下载
预览
Top