Given an algorithmic predictor that is "fair" on some source distribution, will it still be fair on an unknown target distribution that differs from the source within some bound? In this paper, we study the transferability of statistical group fairness for machine learning predictors (i.e., classifiers or regressors) subject to bounded distribution shifts. Such shifts may be introduced by initial training data uncertainties, user adaptation to a deployed predictor, dynamic environments, or the use of pre-trained models in new settings. Herein, we develop a bound that characterizes such transferability, flagging potentially inappropriate deployments of machine learning for socially consequential tasks. We first develop a framework for bounding violations of statistical fairness subject to distribution shift, formulating a generic upper bound for transferred fairness violations as our primary result. We then develop bounds for specific worked examples, focusing on two commonly used fairness definitions (i.e., demographic parity and equalized odds) and two classes of distribution shift (i.e., covariate shift and label shift). Finally, we compare our theoretical bounds to deterministic models of distribution shift and against real-world data, finding that we are able to estimate fairness violation bounds in practice, even when simplifying assumptions are only approximately satisfied.
翻译:如果算法预测在某种源的分布上是“公平”的,那么对于一个与某种源的源码不同、与某些源码不同的未知的目标分布是否公平?在本文中,我们研究受约束分布变化约束的机器学习预测器(即分类器或递减器)统计群体公平性可转让性是否可转让性。这种转变可以通过初始培训数据不确定性、用户适应部署的预测器、动态环境或新环境下使用预先培训的模型来引入。在这里,我们开发了一个界限,将这种可转让性定性为可能不适当的机器学习用于社会影响的任务。我们首先开发了一个框架,将违反统计公平性的行为与分配变化相挂钩,为转移的公平性违反行为制定通用上限,作为我们的主要结果。然后我们为具体工作范例制定界限,侧重于两种常用的公平性定义(即人口均等和均等率)和两种分配变化类型(即变换和标签转换)。最后,我们将理论界限与分配变化的确定性模式和真实世界数据进行比较。我们发现,即使我们只满足了公平性假设,但我们也只能简化了惯例。