Image clustering is an important, and open challenge task in computer vision. Although many methods have been proposed to solve the image clustering task, they only explore images and uncover clusters according to the image features, thus are unable to distinguish visually similar but semantically different images. In this paper, we propose to investigate the task of image clustering with the help of visual-language pre-training model. Different from the zero-shot setting in which the class names are known, we only know the number of clusters in this setting. Therefore, how to map images to a proper semantic space and how to cluster images from both image and semantic spaces are two key problems. To solve the above problems, we propose a novel image clustering method guided by the visual-language pre-training model CLIP, named as \textbf{Semantic-enhanced Image Clustering (SIC)}. In this new method, we propose a method to map the given images to a proper semantic space first and efficient methods to generate pseudo-labels according to the relationships between images and semantics. Finally, we propose to perform clustering with the consistency learning in both image space and semantic space, in a self-supervised learning fashion. Theoretical result on convergence analysis shows that our proposed method can converge in sublinear speed. Theoretical analysis on expectation risk also shows that we can reduce the expectation risk by improving the neighborhood consistency or prediction confidence or reducing neighborhood imbalance. Experimental results on five benchmark datasets clearly show the superiority of our new method.
翻译:在计算机视野中,图像分组是一项重要且公开的挑战任务。 虽然许多方法都建议了解决图像分组任务, 但是它们只是根据图像特性探索图像并发现群集, 因此无法区分视觉上相似但语义上不同的图像。 在本文中, 我们提议在视觉语言培训前模式的帮助下, 调查图像分组任务。 不同于已知类名称的零点设置, 我们只知道在这个环境中的群集数量。 因此, 如何将图像映射到一个合适的语义空间, 以及如何将图像和语义空间的图像分组为两大问题。 为了解决上述问题, 我们提议在视觉语言前培训模式CLIP的指导下, 以视觉语言前模式CLIP 为指导, 名为 textb{ 语言强化图像分组模式( SICT) 。 在这种新方法中, 我们建议了一种方法, 将给定的图像映射到一个正确的语义空间空间空间空间空间空间空间和语义空间空间空间空间图像和语义空间空间图像关系之间的假标签是两大问题。 最后, 我们提议在图像和语义分析中进行分组分组分组与一致性的分组分组,, 学习关于空间正统化结果中, 的自我定位分析中, 显示我们所选取的预结果的预结果分析,, 的预结果,, 将显示的预结果分析中, 。