Spiking neural networks (SNNs) are bio-inspired neural networks with asynchronous discrete and sparse characteristics, which have increasingly manifested their superiority in low energy consumption. Recent research is devoted to utilizing spatio-temporal information to directly train SNNs by backpropagation. However, the binary and non-differentiable properties of spike activities force directly trained SNNs to suffer from serious gradient vanishing and network degradation, which greatly limits the performance of directly trained SNNs and prevents them from going deeper. In this paper, we propose a multi-level firing (MLF) method based on the existing spatio-temporal back propagation (STBP) method, and spiking dormant-suppressed residual network (spiking DS-ResNet). MLF enables more efficient gradient propagation and the incremental expression ability of the neurons. Spiking DS-ResNet can efficiently perform identity mapping of discrete spikes, as well as provide a more suitable connection for gradient propagation in deep SNNs. With the proposed method, our model achieves superior performances on a non-neuromorphic dataset and two neuromorphic datasets with much fewer trainable parameters and demonstrates the great ability to combat the gradient vanishing and degradation problem in deep SNNs.
翻译:Spik神经网络(SNNS)是生动的神经网络,具有非同步、离散和稀疏的特点,这些网络在低能消耗方面日益表现出其优越性。最近的研究致力于利用时空空间信息直接通过反射来训练SNNS。但是,峰值活动的二进制和无区别性质迫使直接训练SNNS遭受严重的梯度消失和网络退化,这极大地限制了直接训练的SNNS的性能,并阻止它们更深地发展。在本文中,我们建议了一种多层次的点火(MLF)方法,其基础是现有的时空反传播(STBP)方法,并用来通过反射对休眠压残余网络(S-ResNet)进行直接培训。MLF使得更高效的梯度传播和神经元的递增表达能力。Spiking DS-ResNet能够有效地进行离心峰峰的特性测绘,并为深层SNNNPS的梯度传播提供更合适的连接。根据拟议方法,我们的模型在不甚深层的神经、甚深层的轨变化的轨道数据系统中实现了高超高水平和极低的磁变变化能力。