Self-supervised contrastive learning offers a means of learning informative features from a pool of unlabeled data. In this paper, we delve into another useful approach -- providing a way of selecting a core-set that is entirely unlabeled. In this regard, contrastive learning, one of a large number of self-supervised methods, was recently proposed and has consistently delivered the highest performance. This prompted us to choose two leading methods for contrastive learning: the simple framework for contrastive learning of visual representations (SimCLR) and the momentum contrastive (MoCo) learning framework. We calculated the cosine similarities for each example of an epoch for the entire duration of the contrastive learning process and subsequently accumulated the cosine-similarity values to obtain the coreset score. Our assumption was that an sample with low similarity would likely behave as a coreset. Compared with existing coreset selection methods with labels, our approach reduced the cost associated with human annotation. The unsupervised method implemented in this study for coreset selection obtained improved results over a randomly chosen subset, and were comparable to existing supervised coreset selection on various classification datasets (e.g., CIFAR, SVHN, and QMNIST).
翻译:自我监督的对比学习提供了一种从一组未贴标签的数据中学习信息特征的方法。 在本文中,我们探索了另一种有用的方法 -- -- 提供了选择完全未贴标签的核心集的方法的一种方法。在这方面,最近提出了对比学习,这是大量自监督方法之一,最近一直提供最高绩效。这促使我们选择了两种对比学习的主要方法:视觉表现对比学习的简单框架(SimCLR)和动力对比学习框架。我们计算了对比学习过程整个期间每个缩格的共生相似性,并随后积累了获得核心集分的相似性值。我们的假设是,与现有核心集选择方法相比,与标签相比,我们的方法降低了与人类标记相关的成本。本研究中采用的未经监督的核心集选择方法在随机选择子集中取得了更好的结果,并可以与现有的受监督的核心集分、S-N-MR-Q等数据分类中的数据集(S-Q、VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-VI-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-III-