ANN-SNN 高精度和超高低低低纬度渗透神经网络优化转换</s> (Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks)

Spiking Neural Networks (SNNs) have gained great attraction due to their distinctive properties of low power consumption and fast inference on neuromorphic hardware. As the most effective method to get deep SNNs, ANN-SNN conversion has achieved comparable performance as ANNs on large-scale datasets. Despite this, it requires long time-steps to match the firing rates of SNNs to the activation of ANNs. As a result, the converted SNN suffers severe performance degradation problems with short time-steps, which hamper the practical application of SNNs. In this paper, we theoretically analyze ANN-SNN conversion error and derive the estimated activation function of SNNs. Then we propose the quantization clip-floor-shift activation function to replace the ReLU activation function in source ANNs, which can better approximate the activation function of SNNs. We prove that the expected conversion error between SNNs and ANNs is zero, enabling us to achieve high-accuracy and ultra-low-latency SNNs. We evaluate our method on CIFAR-10/100 and ImageNet datasets, and show that it outperforms the state-of-the-art ANN-SNN and directly trained SNNs in both accuracy and time-steps. To the best of our knowledge, this is the first time to explore high-performance ANN-SNN conversion with ultra-low latency (4 time-steps). Code is available at https://github.com/putshua/SNN\_conversion\_QCFS

翻译：Spik Spik Neural Networks(SNN)因其低电耗和神经变异硬件快速推导的特殊性能而获得很大的吸引力。作为获取深度 SNNSs的最有效方法,ANNN-SNN转换在大型数据集中作为ANNNSs实现了可比的性能。尽管如此,这需要很长的时间步骤才能匹配SNNS的点火率以激活ANNNSs。因此,转换的SNNN面临严重的性能退化问题,时间步很短,这阻碍了SNNNS的实用应用。在本文件中,我们理论上分析了ANNN-SNNN的转换错误,并得出了SNNNNNS的估计启动功能。然后,我们提议在源 ANNS 中以四分制的点点点点点点点点点化点化点化,这可以更接近SNNNS的启动功能。我们证明, SNNNSs和ANS ERS 数据库的高级时间-时间-10100 和图像S 数据转换规则显示这个高端-NNNNNS 的高级的高级数据。我们用的时间-NUR-ral-ral-ral-ral-rations-al-l-l-l-l-l-l-l-l-l-l-al-l-al-al-l-l-l-al-al-al-al-al-d-ds-ds-destruts disal-l-l-l-ds-lations-lations-l-l-l-l-l-l-l-s-s-s-l-l-l-s-s-s-s-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-set-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-l-</s>

相关内容

激活函数

关注 44

在人工神经网络中，给定一个输入或一组输入，节点的激活函数定义该节点的输出。一个标准集成电路可以看作是一个由激活函数组成的数字网络，根据输入的不同，激活函数可以是开(1)或关(0)。这类似于神经网络中的线性感知器的行为。然而，只有非线性激活函数允许这样的网络只使用少量的节点来计算重要问题，并且这样的激活函数被称为非线性。

【干货书】数据分析优化，Optimization for Modern Data Analysis，117页pdf

专知会员服务

60+阅读 · 2023年2月15日

ICLR 2022杰出论文公布：7篇论文获得，清华朱军课题组摘得

专知会员服务

59+阅读 · 2022年4月22日

高效可扩展图神经网络的研究进展，Recent Advances in Efficient and Scalable Graph Neural Networks

专知会员服务

75+阅读 · 2022年3月15日

一份简单《图神经网络》教程，28页ppt

专知会员服务

124+阅读 · 2020年8月2日