This paper studies the prediction of a target $\mathbf{z}$ from a pair of random variables $(\mathbf{x},\mathbf{y})$, where the ground-truth predictor is additive $\mathbb{E}[\mathbf{z} \mid \mathbf{x},\mathbf{y}] = f_\star(\mathbf{x}) +g_{\star}(\mathbf{y})$. We study the performance of empirical risk minimization (ERM) over functions $f+g$, $f \in \mathcal{F}$ and $g \in \mathcal{G}$, fit on a given training distribution, but evaluated on a test distribution which exhibits covariate shift. We show that, when the class $\mathcal{F}$ is "simpler" than $\mathcal{G}$ (measured, e.g., in terms of its metric entropy), our predictor is more resilient to \emph{heterogenous covariate shifts} in which the shift in $\mathbf{x}$ is much greater than that in $\mathbf{y}$. These results rely on a novel H\"older style inequality for the Dudley integral which may be of independent interest. Moreover, we corroborate our theoretical findings with experiments demonstrating improved resilience to shifts in "simpler" features across numerous domains.
翻译:本文研究一个目标$mathbf{z{z} $的预测, 由一对随机变量$(\\mathbf{x},\mathbf{y}$) 来预测 目标$mathb{E}[\mathbf{z} = mid\ mathbf{x},\mathbf{z{z} = fstar(\mathb{x}) +g{star} (\mathb{star}} (custrib{yf{yf{y}}) 。 我们研究实证风险最小化(ERM) 在函数 $+g$, $f\ g= g$, $f\ g} 和 $g\\ mathb{G} 的预测值, 适合给定的培训分布, 但是在显示变换的测试分配值上, $mathcretary{F} 显示, 当类是“ 简单化” 而不是 美元=regregal develyal_ rudeal_</s>