While the backpropagation of error algorithm enables deep neural network training, it implies (i) bidirectional synaptic weight transport and (ii) update locking until the forward and backward passes are completed. Not only do these constraints preclude biological plausibility, but they also hinder the development of low-cost adaptive smart sensors at the edge, as they severely constrain memory accesses and entail buffering overhead. In this work, we show that the one-hot-encoded labels provided in supervised classification problems, denoted as targets, can be viewed as a proxy for the error sign. Therefore, their fixed random projections enable a layerwise feedforward training of the hidden layers, thus solving the weight transport and update locking problems while relaxing the computational and memory requirements. Based on these observations, we propose the direct random target projection (DRTP) algorithm and demonstrate that it provides a tradeoff between accuracy and computational cost that is suitable for adaptive edge computing devices.
翻译:虽然错误算法的反向调整可以进行深神经网络培训,但这意味着 (一) 双向合成重量迁移和(二) 更新锁定,直到前向和后向通道完成。这些限制不仅排除了生物的可信度,而且阻碍边缘低成本适应性智能传感器的开发,因为这些限制严重限制记忆存取,并导致缓冲间接费用。在这项工作中,我们显示,在监督分类问题中提供的单热编码标签,称为目标,可以被视为错误标志的代用。因此,其固定随机预测使得对隐藏层进行分层进料前方培训,从而解决重量传输和更新锁定问题,同时放松计算和记忆要求。我们根据这些观察,提出直接随机目标预测算法,并表明它提供了精确度和计算成本之间的权衡,适合适应边缘计算设备。