Making fair decisions is crucial to ethically implementing machine learning algorithms in social settings. In this work, we consider the celebrated definition of counterfactual fairness [Kusner et al., NeurIPS, 2017]. We begin by showing that an algorithm which satisfies counterfactual fairness also satisfies demographic parity, a far simpler fairness constraint. Similarly, we show that all algorithms satisfying demographic parity can be trivially modified to satisfy counterfactual fairness. Together, our results indicate that counterfactual fairness is basically equivalent to demographic parity, which has important implications for the growing body of work on counterfactual fairness. We then validate our theoretical findings empirically, analyzing three existing algorithms for counterfactual fairness against three simple benchmarks. We find that two simple benchmark algorithms outperform all three existing algorithms -- in terms of fairness, accuracy, and efficiency -- on several data sets. Our analysis leads us to formalize a concrete fairness goal: to preserve the order of individuals within protected groups. We believe transparency around the ordering of individuals within protected groups makes fair algorithms more trustworthy. By design, the two simple benchmark algorithms satisfy this goal while the existing algorithms for counterfactual fairness do not.
翻译:做出公正决定对于在社会环境中实施机器学习算法在道德上是关键。 在这项工作中,我们考虑了反事实公平值得庆幸的定义[Kusner 等人, NeurIPS, 2017]。我们首先表明,符合反事实公平的一种算法也满足了人口均等,这是一个简单得多的公平限制。同样,我们表明,所有满足人口均等的算法都可能被轻描淡写地修改,以满足反事实公平。我们的共同结果表明,反事实公平基本上等同于人口均等,这对不断增长的反事实公平工作具有重要影响。然后,我们从经验上验证了我们的理论结论,根据三个简单的基准分析了三种现有的反事实公平。我们发现,两种简单的基准算法在一些数据集上比所有三种现有算法 -- -- 从公平、准确性和效率上看 -- -- 优于所有三种现有算法。我们的分析引导我们正式确定一个具体的公平目标:维护受保护群体内部个人秩序。我们认为,围绕受保护群体内部个人秩序的透明度使公平算法更加可信。通过设计,两种简单的基准算法满足了这一目标,而现有的反事实公平性算法则并非如此。