Bio-inspired spiking neural networks (SNNs), operating with asynchronous binary signals (or spikes) distributed over time, can potentially lead to greater computational efficiency on event-driven hardware. The state-of-the-art SNNs suffer from high inference latency, resulting from inefficient input encoding, and sub-optimal settings of the neuron parameters (firing threshold, and membrane leak). We propose DIET-SNN, a low-latency deep spiking network that is trained with gradient descent to optimize the membrane leak and the firing threshold along with other network parameters (weights). The membrane leak and threshold for each layer of the SNN are optimized with end-to-end backpropagation to achieve competitive accuracy at reduced latency. The analog pixel values of an image are directly applied to the input layer of DIET-SNN without the need to convert to spike-train. The first convolutional layer is trained to convert inputs into spikes where leaky-integrate-and-fire (LIF) neurons integrate the weighted inputs and generate an output spike when the membrane potential crosses the trained firing threshold. The trained membrane leak controls the flow of input information and attenuates irrelevant inputs to increase the activation sparsity in the convolutional and dense layers of the network. The reduced latency combined with high activation sparsity provides large improvements in computational efficiency. We evaluate DIET-SNN on image classification tasks from CIFAR and ImageNet datasets on VGG and ResNet architectures. We achieve top-1 accuracy of 69% with 5 timesteps (inference latency) on the ImageNet dataset with 12x less compute energy than an equivalent standard ANN. Additionally, DIET-SNN performs 20-500x faster inference compared to other state-of-the-art SNN models.
翻译:由生物启发的神经神经网络(SNNS)运行的神经神经网络(SNNS),是一个低纬度的深度神经网络(SNNS),其使用时间分布的二进制二进制信号(或峰值),有可能提高事件驱动硬件的计算效率。最先进的SNNS由于输入编码效率低下,神经参数参数的精度延迟度高,以及次最佳设置(火线阈值和膜膜泄漏)。我们建议DIET-SNNN,这是一个低纬度的深度神经网络网络网络网络,该网络的精度深度精度下降,以优化离子膜泄漏和发射阈值与其他网络参数(重量)的离子网络启动。SNBRBRNE泄漏和电路流中经过训练的神经中,Slickral-lickral-lickral 数据流中,Slickrickral-lickral-lickral-lickral-lickral-lickral-lickral-lickral-licks) 数据流。