Self-supervised learning for image representations has recently had many breakthroughs with respect to linear evaluation and fine-tuning evaluation. These approaches rely on both cleverly crafted loss functions and training setups to avoid the feature collapse problem. In this paper, we improve on the recently proposed VICReg paper, which introduced a loss function that does not rely on specialized training loops to converge to useful representations. Our method improves on a covariance term proposed in VICReg, and in addition we augment the head of the architecture by an IterNorm layer that greatly accelerates convergence of the model. Our model achieves superior performance on linear evaluation and fine-tuning evaluation on a subset of the UCR time series classification archive and the PTB-XL ECG dataset.
翻译:最近,在线性评价和微调评价方面,自我监督的图像表现学习取得了许多突破,这些办法既依靠巧妙设计的损失功能,又依靠培训设置,以避免特征崩溃问题。在本文件中,我们改进了最近提出的国际中心区域局文件,其中引入了一种不依赖专门培训循环来汇集有用的表示方式的损失功能。我们的方法在维也纳国际中心区域局提出的一个共变术语上有所改进,此外,我们用一个能大大加速模型趋同的IterNorm层来扩大结构的首级。我们的模型在线性评价和微调评价中取得了优异的成绩,对UCR时间序列档案和PTB-XL ECG数据集的一组进行了微调评价。