Semi-supervised learning (SSL) provides an effective means of leveraging unlabelled data to improve a model performance. Even though the domain has received a considerable amount of attention in the past years, most methods present the common drawback of lacking theoretical guarantees. Our starting point is to notice that the estimate of the risk that most discriminative SSL methods minimise is biased, even asymptotically. This bias impedes the use of standard statistical learning theory and can hurt empirical performance. We propose a simple way of removing the bias. Our debiasing approach is straightforward to implement and applicable to most deep SSL methods. We provide simple theoretical guarantees on the trustworthiness of these modified methods, without having to rely on the strong assumptions on the data distribution that SSL theory usually requires. In particular, we provide generalisation error bounds for the proposed methods. We evaluate debiased versions of different existing SSL methods, such as the Pseudo-label method and Fixmatch, and show that debiasing can compete with classic deep SSL techniques in various settings by providing better calibrated models. Additionally, we provide a theoretical explanation of the intuition of the popular SSL methods.
翻译:半监督的学习(SSL)提供了一种有效的手段来利用未贴标签的数据来改进模型性能。尽管过去几年,这个领域得到了相当程度的注意,但大多数方法都是缺乏理论保证的共同缺点。我们的出发点是注意,对最歧视性的SSL方法最小化的风险的估计是偏差的,即使只是轻率的。这种偏差妨碍了标准统计学习理论的使用,并可能损害经验性能。我们提出了一个消除偏差的简单方法。我们的偏差方法可以直接地实施并适用于最深的SSL方法。我们对这些修改过的方法的可靠性提供了简单的理论保证,而不必依赖关于SSL理论通常需要的数据分配的强有力的假设。特别是,我们提供了拟议方法的概括性误差。我们评估了不同的现有SSL方法的偏差版本,例如Pseudo-标签方法和Fixmatch,并表明,通过提供更好的校准模型,我们从理论上解释了普尔夫·SSL方法的直率。</s>