It is well established in neuroscience that color vision plays an essential part in the human visual perception system. Meanwhile, many novel designs for computer vision inspired by human vision have achieved success in a wide range of tasks and applications. Nonetheless, how color differences affect machine vision has not been well explored. Our work tries to bridge this gap between the human color vision aspect of visual recognition and that of the machine. To achieve this, we curate two datasets: CIFAR10-F and CIFAR100-F, which are based on the foreground colors of the popular CIFAR datasets. Together with CIFAR10-B and CIFAR100-B, the existing counterpart datasets with information on the background colors of CIFAR test sets, we assign each image based on its color contrast level per its foreground and background color labels and use this as a proxy to study how color contrast affects machine vision. We first conduct a proof-of-concept study, showing the effect of color difference and validate our datasets. Furthermore, on a broader level, an important characteristic of human vision is its robustness against ambient changes; therefore, drawing inspirations from ophthalmology and the robustness literature, we analogize contrast sensitivity from the human visual aspect to machine vision and complement the current robustness study using corrupted images with our CIFAR-CoCo datasets. In summary, motivated by neuroscience and equipped with the datasets we curate, we devise a new framework in two dimensions to perform extensive analyses on the effect of color contrast and corrupted images: (1) model architecture, (2) model size, to measure the perception ability of machine vision beyond total accuracy. We also explore how task complexity and data augmentation play a role in this setup. Our results call attention to new evaluation approaches for human-like machine perception.
翻译:在神经科学中,色观在人类视觉认知系统中起着不可或缺的作用。 同时,许多由人类视觉启发的计算机视觉新设计在广泛的任务和应用中取得了成功。 尽管如此,对颜色差异如何影响机器视觉还没有很好地探索。 我们的工作试图缩小视觉认知中的人类颜色视觉方面与机器的视觉方面之间的差距。 为了做到这一点,我们首先进行两个数据集:CIFAR10-F 和 CIFAR100-F, 其基础是广受欢迎的CIFAR 图像的表面颜色。同时,许多由人类视觉启发的计算机视觉新设计在一系列任务和应用中取得了成功。 与CIFAR10-B 和 CIFAR100-B 一起, 现有的对应数据集在CIFAR 测试组的背景颜色信息上取得了成功。 然而,我们根据每个图像的颜色对比水平来分配每个图像。 为了研究颜色对比对机器视觉的视觉影响。 我们首先进行检测测试,显示颜色差异的影响,并验证我们的数据集。 在更广泛的层次上, 人类视觉的一个重要特征是它的坚固性 相对于环境变化的变化变化; 因此,我们用模拟的视觉感觉分析中, 我们的视觉结构的感光观分析, 和视觉结构的视觉分析, 我们的视觉分析, 我们的视觉分析, 分析中, 分析, 的视觉结构的视觉分析, 分析。