Spatio-temporal pattern recognition is a fundamental ability of the brain which is required for numerous real-world activities. Recent deep learning approaches have reached outstanding accuracies in such tasks, but their implementation on conventional embedded solutions is still very computationally and energy expensive. Tactile sensing in robotic applications is a representative example where real-time processing and energy efficiency are required. Following a brain-inspired computing approach, we propose a new benchmark for spatio-temporal tactile pattern recognition at the edge through Braille letter reading. We recorded a new Braille letters dataset based on the capacitive tactile sensors of the iCub robot's fingertip. We then investigated the importance of spatial and temporal information as well as the impact of event-based encoding on spike-based computation. Afterward, we trained and compared feedforward and recurrent Spiking Neural Networks (SNNs) offline using Backpropagation Through Time (BPTT) with surrogate gradients, then we deployed them on the Intel Loihi neuromorphic chip for fast and efficient inference. We compared our approach to standard classifiers, in particular to the Long Short-Term Memory (LSTM) deployed on the embedded NVIDIA Jetson GPU, in terms of classification accuracy, power, energy consumption, and delay. Our results show that the LSTM reaches ~97% of accuracy, outperforming the recurrent SNN by ~17% when using continuous frame-based data instead of event-based inputs. However, the recurrent SNN on Loihi with event-based inputs is ~500 times more energy-efficient than the LSTM on Jetson, requiring a total power of only ~30 mW. This work proposes a new benchmark for tactile sensing and highlights the challenges and opportunities of event-based encoding, neuromorphic hardware, and spike-based computing for spatio-temporal pattern recognition at the edge.
翻译:Spatio- 时空模式识别是大脑的一种基本能力,这是许多真实世界活动所需要的。 最近深层次的学习方法在这类任务中达到了杰出的触觉, 但对于常规嵌入解决方案的实施仍然非常具有计算性和能源成本。 机器人应用中的触摸感是需要实时处理和能源效率的一个具有代表性的范例。 在采用大脑启发的计算方法之后, 我们建议了一个新的标准, 在边缘点对时空模式进行触觉识别。 我们记录了一个基于iCub机器人直指事件的电动传感器的经常性盲文数据集。 我们记录了一个基于iCub 的电动触摸感应传感器的新盲文数据集。 我们随后调查了空间和时间信息信息的重要性以及基于事件的编码对加固计算的影响。 之后,我们培训和比较了向上和反复的Spik Neural网络(SNNNN)使用基于时间的反光调梯度梯度梯度信号, 然后我们用Inteforal- IM 快速和高效的神经失常状态传感器输入数据。 我们比较了S- Nation- NL- sal- slial- dalalal- dalalal- disal- disal- 时间, 我们的S- dental- disal- 时间对S- disal- disal- dislational- 需要一个基于S- sal- dental- dental- dental- sutal- 的S- 的S- 时间的S- sal- dislental- sildal- sutaltrade- dental- dent- sutaldal- 机算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算算值比值比值比值比比比比比比。