Color and structure are the two pillars that construct an image. Usually, the structure is well expressed through a rich spectrum of colors, allowing objects in an image to be recognized by neural networks. However, under extreme limitations of color space, the structure tends to vanish, and thus a neural network might fail to understand the image. Interested in exploring this interplay between color and structure, we study the scientific problem of identifying and preserving the most informative image structures while constraining the color space to just a few bits, such that the resulting image can be recognized with possibly high accuracy. To this end, we propose a color quantization network, ColorCNN, which learns to structure the images from the classification loss in an end-to-end manner. Given a color space size, ColorCNN quantizes colors in the original image by generating a color index map and an RGB color palette. Then, this color-quantized image is fed to a pre-trained task network to evaluate its performance. In our experiment, with only a 1-bit color space (i.e., two colors), the proposed network achieves 82.1% top-1 accuracy on the CIFAR10 dataset, outperforming traditional color quantization methods by a large margin. For applications, when encoded with PNG, the proposed color quantization shows superiority over other image compression methods in the extremely low bit-rate regime. The code is available at: https://github.com/hou-yz/color_distillation.
翻译:颜色和结构是构建图像的两大支柱。 通常, 结构会通过丰富的颜色频谱来清晰表达, 允许神经网络识别图像中的物体。 但是, 在色彩空间的极端限制下, 结构会消失, 因而神经网络可能无法理解图像 。 我们有兴趣探索颜色和结构之间的这种相互作用, 我们研究在将最丰富的图像结构限制在几个位子上同时识别和保存最丰富的图像结构的科学问题, 从而能够以可能高的精确度来识别由此生成的图像。 为此, 我们建议了一个颜色定量化网络, 颜色CNN, 以端到端的方式学习从分类损失中构建图像。 鉴于颜色空间的极端限制, 颜色CNN 将原始图像的颜色量化为颜色和结构之间的相互作用。 然后, 这个色彩定量化图像被反馈到一个经过事先训练的任务网络来评估其性能。 在我们的实验中, 只有一个比位空间( e. two colorizion), 拟议的网络将82. 1% 的颜色- blority ad- procial adal adalalizal digistration subal ex ex ex ex exupal ex laudal laus ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex suolololololus ex ex ex ex ex ex ex ex explus ex ex ex ex ex exual ex ex ex ex ex ex exubolual ex ex subaltial explut exal exal ex ex ex ex ex ex ex ex ex ex ex ex exubaltialtialalalalalalalalalalal ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex ex