We consider unsupervised domain adaptation (UDA), where labeled data from a source domain (e.g., photographs) and unlabeled data from a target domain (e.g., sketches) are used to learn a classifier for the target domain. Conventional UDA methods (e.g., domain adversarial training) learn domain-invariant features to improve generalization to the target domain. In this paper, we show that contrastive pre-training, which learns features on unlabeled source and target data and then fine-tunes on labeled source data, is competitive with strong UDA methods. However, we find that contrastive pre-training does not learn domain-invariant features, diverging from conventional UDA intuitions. We show theoretically that contrastive pre-training can learn features that vary subtantially across domains but still generalize to the target domain, by disentangling domain and class information. Our results suggest that domain invariance is not necessary for UDA. We empirically validate our theory on benchmark vision datasets.
翻译:我们认为,在未经监督的域适应(UDA)中,来自源域的标签数据(如照片)和来自目标域的未标签数据(如草图)被用于学习目标域的分类器(如草图),常规UDA方法(如域对战训练)学习域异性特征,以改进目标域的概括化。在本文中,我们显示,对比式的预培训,即学习未标签源和目标数据特征,然后对标签源数据的微调,与强有力的UDA方法具有竞争力。然而,我们发现,对比式的预培训并不学习域异性特征,与传统的UDA直觉不同。我们从理论上表明,对比式的预培训可以学习不同领域分化但通过脱钩域和类信息仍普遍化到目标域的特征。我们的结果显示,UDA没有必要使用网域变量。我们从经验上验证了我们关于基准愿景数据集的理论。