Neuromorphic (event-based) image sensors draw inspiration from the human-retina to create an electronic device that can process visual stimuli in a way that closely resembles its biological counterpart. These sensors process information significantly different than the traditional RGB sensors. Specifically, the sensory information generated by event-based image sensors are orders of magnitude sparser compared to that of RGB sensors. The first generation of neuromorphic image sensors, Dynamic Vision Sensor (DVS), are inspired by the computations confined to the photoreceptors and the first retinal synapse. In this work, we highlight the capability of the second generation of neuromorphic image sensors, Integrated Retinal Functionality in CMOS Image Sensors (IRIS), which aims to mimic full retinal computations from photoreceptors to output of the retina (retinal ganglion cells) for targeted feature-extraction. The feature of choice in this work is Object Motion Sensitivity (OMS) that is processed locally in the IRIS sensor. Our results show that OMS can accomplish standard computer vision tasks with similar efficiency to conventional RGB and DVS solutions but offers drastic bandwidth reduction. This cuts the wireless and computing power budgets and opens up vast opportunities in high-speed, robust, energy-efficient, and low-bandwidth real-time decision making.
翻译:物体运动敏感性:生物启发的解决方案——针对事件相机的自我运动问题
神经形态学(事件)图像传感器从人类视网膜汲取灵感,创造了一种可以以与生物同样方式处理视觉刺激的电子设备。这些传感器的信息处理方式与传统的RGB传感器显著不同。具体而言,由事件型图像传感器产生的感知信息比RGB传感器稀疏多个数量级。神经形态学图像传感器的第一代,动态视觉传感器(DVS),受到了光感受器和视网膜第一突触中的计算的启发。在这项工作中,我们强调了神经形态学图像传感器第二代——集成的CMOS图像传感器网膜功能(IRIS)的能力,它旨在模仿光感受器到视网膜输出(网膜节细胞)的完整视网膜计算,以进行目标特征提取。本文中所选的特征是物体运动敏感性(OMS),该特征在IRIS传感器中以本地方式处理。我们的结果表明,OMS可以以类似于传统RGB和DVS解决方案的效率完成标准计算机视觉任务,但可以大幅减少带宽。这可以削减无线和计算功率预算,并在高速、强韧、节能、低带宽实时决策方面提供广泛的机会。