Neuromorphic (event-based) image sensors draw inspiration from the human-retina to create an electronic device that can process visual stimuli in a way that closely resembles its biological counterpart. These sensors process information significantly different than the traditional RGB sensors. Specifically, the sensory information generated by event-based image sensors are orders of magnitude sparser compared to that of RGB sensors. The first generation of neuromorphic image sensors, Dynamic Vision Sensor (DVS), are inspired by the computations confined to the photoreceptors and the first retinal synapse. In this work, we highlight the capability of the second generation of neuromorphic image sensors, Integrated Retinal Functionality in CMOS Image Sensors (IRIS), which aims to mimic full retinal computations from photoreceptors to output of the retina (retinal ganglion cells) for targeted feature-extraction. The feature of choice in this work is Object Motion Sensitivity (OMS) that is processed locally in the IRIS sensor. We study the capability of OMS in solving the ego-motion problem of the event-based cameras. Our results show that OMS can accomplish standard computer vision tasks with similar efficiency to conventional RGB and DVS solutions but offers drastic bandwidth reduction. This cuts the wireless and computing power budgets and opens up vast opportunities in high-speed, robust, energy-efficient, and low-bandwidth real-time decision making.
翻译:Abstract: 神经形态学(事件)图像传感器从人类视网膜中汲取灵感,创造了一种可以以与其生物对应物相似的方式处理视觉刺激的电子设备。这些传感器处理的信息与传统的RGB传感器相比显著不同。具体而言,事件型图像传感器生成的感官信息比RGB传感器稀疏几个数量级。第一代神经形态学图像传感器——动态视觉传感器(DVS)——受到限于光感受器和视网膜神经元之间第一个突触的计算的启发。在本文中,我们强调了第二代神经形态学图像传感器——集成视网膜功能CMOS图像传感器(IRIS)的能力,它旨在模仿从光感受器到视网膜输出(视网膜神经元)的全面视网膜计算,以进行有针对性的特征提取。本研究选择的特征是目标运动敏感度(OMS),该特征在IRIS传感器中局部处理。我们研究了OMS在解决事件型摄像机自运动问题方面的能力。我们的结果表明,OMS可以像传统的RGB和DVS方案一样有效地完成标准计算机视觉任务,但可大大减少带宽。这减小了无线和计算功率预算,并在高速、强韧、节能和低带宽实时决策方面开辟了广阔的机会。