Learning a domain-invariant representation has become one of the most popular approaches for domain adaptation/generalization. In this paper, we show that the invariant representation may not be sufficient to guarantee a good generalization, where the labeling function shift should be taken into consideration. Inspired by this, we first derive a new generalization upper bound on the empirical risk that explicitly considers the labeling function shift. We then propose Domain-specific Risk Minimization (DRM), which can model the distribution shifts of different domains separately and select the most appropriate one for the target domain. Extensive experiments on four popular domain generalization datasets, CMNIST, PACS, VLCS, and DomainNet, demonstrate the effectiveness of the proposed DRM for domain generalization with the following advantages: 1) it significantly outperforms competitive baselines; 2) it enables either comparable or superior accuracies on all training domains comparing to vanilla empirical risk minimization (ERM); 3) it remains very simple and efficient during training, and 4) it is complementary to invariant learning approaches.
翻译:在本文中,我们表明,变量代表性可能不足以保证良好的概括化,因为标签功能的转变应当予以考虑。受此启发,我们首先对明确考虑标签功能变化的经验风险得出一个新的概括性上限。我们然后提议具体领域的风险最小化(DRM),它可以分别模拟不同领域的分布转移,并为目标领域选择最合适的数据。关于四个广域通用数据集(CMNIST、PACS、VLCS和DomainNet)的广泛实验表明拟议的DRM对域化的有效性,其优点如下:(1) 它大大超过竞争性基准;(2) 它使得所有培训领域的可比或高级理解与香草实验风险最小化相比(ERM);(3) 它在培训期间仍然非常简单和有效,(4) 它补充了差异性学习方法。