Spatio-temporal pattern recognition is a fundamental ability of the brain which is required for numerous real-world applications. Recent deep learning approaches have reached outstanding accuracy in such tasks, but their implementation on conventional embedded solutions is still very computationally and energy expensive. Tactile sensing in robotic applications is a representative example where real-time processing and energy-efficiency are required. Following a brain-inspired computing approach, we propose a new benchmark for spatio-temporal tactile pattern recognition at the edge through braille letters reading. We recorded a new braille letters dataset based on the capacitive tactile sensors/fingertip of the iCub robot, then we investigated the importance of temporal information and the impact of event-based encoding for spike-based/event-based computation. Afterwards, we trained and compared feed-forward and recurrent spiking neural networks (SNNs) offline using back-propagation through time with surrogate gradients, then we deployed them on the Intel Loihi neuromorphic chip for fast and efficient inference. We confronted our approach to standard classifiers, in particular to a Long Short-Term Memory (LSTM) deployed on the embedded Nvidia Jetson GPU in terms of classification accuracy, power/energy consumption and computational delay. Our results show that the LSTM outperforms the recurrent SNN in terms of accuracy by 14%. However, the recurrent SNN on Loihi is 237 times more energy-efficient than the LSTM on Jetson, requiring an average power of only 31mW. This work proposes a new benchmark for tactile sensing and highlights the challenges and opportunities of event-based encoding, neuromorphic hardware and spike-based computing for spatio-temporal pattern recognition at the edge.
翻译:空间时空模式的识别是大脑的一种基本能力,这是许多真实世界应用所需要的。 最近的深层次学习方法在这类任务中已经达到杰出的准确性, 但对于常规嵌入解决方案的实施仍然非常具有计算性和能源成本。 机器人应用中的触觉感是需要实时处理和能源效率的一个具有代表性的范例。 通过由大脑启发的计算方法, 我们提出了一个在边缘通过直线读读读读读来测量时触摸时的触觉模式识别的新基准。 我们根据iCub 机器人的经常电动感应传感器/定点仪记录了一个新的直线字母数据集数据集。 然后我们调查了时间信息的重要性以及基于事件编码的编码对基于峰值/日志的计算的影响。 之后,我们训练并比较了向上和反复的神经网络(SNNNNS), 然后我们通过S- Rial-S 的系统平均智能变异性变压速度的变现, 也就是S- mal- mal- dislal- IM 的变压, 在S- sal- deal- sal- ladeal- laxeral ladeal ladeal lax 上,我们用S- sleval- slational- sleval 的算算算算算算算算算算算算算算出了一个比 和S- s- s- slental- slental- s- s- s- s- s- slentalentalentalental- stral- slental- stravaldaltravaldaltrade 的变算算算算算算算算算算算算法, 。