From human physiology to environmental evolution, important processes in nature often exhibit meaningful and strong periodic or quasi-periodic changes. Due to their inherent label scarcity, learning useful representations for periodic tasks with limited or no supervision is of great benefit. Yet, existing self-supervised learning (SSL) methods overlook the intrinsic periodicity in data, and fail to learn representations that capture periodic or frequency attributes. In this paper, we present SimPer, a simple contrastive SSL regime for learning periodic information in data. To exploit the periodic inductive bias, SimPer introduces customized augmentations, feature similarity measures, and a generalized contrastive loss for learning efficient and robust periodic representations. Extensive experiments on common real-world tasks in human behavior analysis, environmental sensing, and healthcare domains verify the superior performance of SimPer compared to state-of-the-art SSL methods, highlighting its intriguing properties including better data efficiency, robustness to spurious correlations, and generalization to distribution shifts. Code and data are available at: https://github.com/YyzHarry/SimPer.
翻译:从人类生理演变到环境演变,大自然中的重要过程往往表现出有意义的和强大的周期性或准周期性的变化。由于它们固有的标签稀缺,学习对定期任务有用且监督有限或没有监督的介绍是大有好处的。然而,现有的自监督学习方法忽略了数据固有的周期性,没有学习能够捕捉周期性或频率性特征的介绍。在本文中,我们介绍SimPer,这是一个简单的对比性SSL制度,用于在数据中学习定期信息。为了利用定期的诱导偏差,SimPer引入了定制的扩增、特征相似性措施以及普遍的对比性损失,用于学习高效和稳健的定期介绍。关于人类行为分析、环境遥感和保健领域共同现实世界任务的广泛实验对SimPer的优异性表现进行了核实,突出了SimPer与最新科学的SSL方法相比的优异性,强调了其令人感兴趣的特性,包括更高的数据效率、对刺激性相关性的强性和对分布变化的概括性。代码和数据见:https://github.com/YyzHarry/SimPer。