Event-based cameras have recently shown great potential for high-speed motion estimation owing to their ability to capture temporally rich information asynchronously. Spiking Neural Networks (SNNs), with their neuro-inspired event-driven processing can efficiently handle such asynchronous data, while neuron models such as the leaky-integrate and fire (LIF) can keep track of the quintessential timing information contained in the inputs. SNNs achieve this by maintaining a dynamic state in the neuron memory, retaining important information while forgetting redundant data over time. Thus, we posit that SNNs would allow for better performance on sequential regression tasks compared to similarly sized Analog Neural Networks (ANNs). However, deep SNNs are difficult to train due to vanishing spikes at later layers. To that effect, we propose an adaptive fully-spiking framework with learnable neuronal dynamics to alleviate the spike vanishing problem. We utilize surrogate gradient-based backpropagation through time (BPTT) to train our deep SNNs from scratch. We validate our approach for the task of optical flow estimation on the Multi-Vehicle Stereo Event-Camera (MVSEC) dataset and the DSEC-Flow dataset. Our experiments on these datasets show an average reduction of 13% in average endpoint error (AEE) compared to state-of-the-art ANNs. We also explore several down-scaled models and observe that our SNN models consistently outperform similarly sized ANNs offering 10%-16% lower AEE. These results demonstrate the importance of SNNs for smaller models and their suitability at the edge. In terms of efficiency, our SNNs offer substantial savings in network parameters (48.3x) and computational energy (10.2x) while attaining ~10% lower EPE compared to the state-of-the-art ANN implementations.
翻译:以事件为基础的相机最近显示出了高速运动估计的巨大潜力,因为它们能够不同步地捕捉时间丰富的信息。 Spiking NealNetworks(SNNS)及其神经引发的事件驱动处理能够有效处理类似同步数据,而像泄漏整合和火灾(LIF)这样的神经模型可以跟踪输入中包含的典型时间信息。 SNNS通过在神经记忆中保持动态状态,保留重要信息,同时随着时间的推移忘记多余的数据。因此,我们假设SNNS能够比类似规模的Analog Neur网络(ANNSs)更好地完成连续回归任务。然而,深SNNNNS(SNN)系统很难被训练,因为随后的层会消失。为此,我们提议一个适应性全面跳动的框架,其中可以学习神经动态动态,以缓解激增的问题。我们用更低的梯度梯度梯度反回流模型(BTTT) 来训练我们更深的SNNNF(SNS) 。我们验证了连续回归的回归参数的运行方法,在SMS-NEVS-IAS平均数据流中显示我们的平均数据流流流流中,这些SNEA-NED-NEA-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S-S</s>