Hierarchical semantic structures naturally exist in an image dataset, in which several semantically relevant image clusters can be further integrated into a larger cluster with coarser-grained semantics. Capturing such structures with image representations can greatly benefit the semantic understanding on various downstream tasks. Existing contrastive representation learning methods lack such an important model capability. In addition, the negative pairs used in these methods are not guaranteed to be semantically distinct, which could further hamper the structural correctness of learned image representations. To tackle these limitations, we propose a novel contrastive learning framework called Hierarchical Contrastive Selective Coding (HCSC). In this framework, a set of hierarchical prototypes are constructed and also dynamically updated to represent the hierarchical semantic structures underlying the data in the latent space. To make image representations better fit such semantic structures, we employ and further improve conventional instance-wise and prototypical contrastive learning via an elaborate pair selection scheme. This scheme seeks to select more diverse positive pairs with similar semantics and more precise negative pairs with truly distinct semantics. On extensive downstream tasks, we verify the superior performance of HCSC over state-of-the-art contrastive methods, and the effectiveness of major model components is proved by plentiful analytical studies. We build a comprehensive model zoo in Sec. D. Our source code and model weights are available at https://github.com/gyfastas/HCSC
翻译:在图像数据集中,自然存在一些具有地震相关性的图像群集,可以进一步将若干具有地震相关性的图像群集整合成一个更大的群集,具有粗糙的语义编码(HCSC),用图像显示这种结构可以极大地促进下游任务中的语义理解。现有的对比性代表性学习方法缺乏如此重要的模型能力。此外,这些方法中使用的负对无法保证其在语义上具有区别性,这可能会进一步妨碍所学图像的结构性正确性。为了解决这些局限性,我们提议了一个新的对比性学习框架,称为“高正对比性选择性选择编码(HCSC)。在这个框架中,建立一套等级原型结构,并动态地更新,以代表潜在空间中的数据所依据的等级性语义结构。为了使图像显示更好地适应这种语义结构,我们采用并进一步改进传统的以实例为根据和准的对比性学习方法,这可能会进一步妨碍所学成的图像显示结构结构的正确性。为了克服这些局限性,我们试图选择具有类似语义性的正面配方言和更精确的负对子,并具有真正不同的定式的Serubus-debal comtial 。在广泛的下游任务中,我们验证了高级/Se-cal-lax-de-lax-lax 。我们用了一种高级的模型分析方法,我们验证了我们进行了一种比较了我们制制制成为主的模型/制的系统。