We show that the effectiveness of the well celebrated Mixup [Zhang et al., 2018] can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss. This simple change not only provides much improved accuracy but also significantly improves the quality of the predictive uncertainty estimation of Mixup in most cases under various forms of covariate shifts and out-of-distribution detection experiments. In fact, we observe that Mixup yields much degraded performance on detecting out-of-distribution samples possibly, as we show empirically, because of its tendency to learn models that exhibit high-entropy throughout; making it difficult to differentiate in-distribution samples from out-distribution ones. To show the efficacy of our approach (RegMixup), we provide thorough analyses and experiments on vision datasets (ImageNet & CIFAR-10/100) and compare it with a suite of recent approaches for reliable uncertainty estimation.
翻译:我们表明,如果人们所熟知的混合[Zhang等人,2018年]能够进一步提高其效力,如果它不是将它作为唯一的学习目标,而是作为标准跨热带作物损失的又一种常规化剂使用。这种简单的改变不仅提高了准确性,而且大大改善了在各种形式的共变转移和分配外检测实验下多数情况下对混合的预测不确定性估计的质量。事实上,我们观察到,混合在检测分配外样本方面产生非常低的性能,正如我们的经验所表明的那样,因为它倾向于学习在整个过程中表现出高持久性的模型;难以区分分配中的样本和分配外的样本。为了展示我们的方法(RegMixup)的功效,我们提供了对视觉数据集(ImageNet & CIFAR-10,100)的透彻分析和实验,并将它与最近一套可靠不确定性估计方法进行比较。