Conventional semi-supervised learning (SSL) methods, e.g., MixMatch, achieve great performance when both labeled and unlabeled dataset are drawn from the same distribution. However, these methods often suffer severe performance degradation in a more realistic setting, where unlabeled dataset contains out-of-distribution (OOD) samples. Recent approaches mitigate the negative influence of OOD samples by filtering them out from the unlabeled data. Our studies show that it is not necessary to get rid of OOD samples during training. On the contrary, the network can benefit from them if OOD samples are properly utilized. We thoroughly study how OOD samples affect DNN training in both low- and high-dimensional spaces, where two fundamental SSL methods are considered: Pseudo Labeling (PL) and Data Augmentation based Consistency Training (DACT). Conclusion is twofold: (1) unlike PL that suffers performance degradation, DACT brings improvement to model performance; (2) the improvement is closely related to class-wise distribution gap between the labeled and the unlabeled dataset. Motivated by this observation, we further improve the model performance by bridging the gap between the labeled and the unlabeled datasets (containing OOD samples). Compared to previous algorithms paying much attention to distinguishing between ID and OOD samples, our method makes better use of OOD samples and achieves state-of-the-art results.
翻译:常规半监督学习方法,例如MixMatch,在从同一分布中提取标签和未贴标签数据集时,能够取得很高的性能;然而,这些方法往往在更现实的环境中出现严重的性能退化,因为未贴标签的数据集含有分配(OOOD)样本。最近采用的方法从未贴标签的数据中过滤OOOD样本,从而减轻OOD样本的消极影响。我们的研究表明,在培训期间没有必要去除OOD样本。相反,如果适当使用OOOD样本,网络就可以从中获益。我们彻底研究OOD样本如何影响低和高维空间的DNN培训,在这种环境中,考虑两种基本的SSL(PL)数据集包含分配(OOD)样本,基于数据放大(DACT)的样本。结论有两重:(1)PL(PL)不同的是,DACT为模型的性能带来改进;(2)改进与标签标签和未贴标签数据集的样本之间的等级分布差距密切相关。我们通过这种观察,将ODG的样品与先前的样品进行更好的比较。