When machine-learning algorithms are deployed in high-stakes decisions, we want to ensure that their deployment leads to fair and equitable outcomes. This concern has motivated a fast-growing literature that focuses on diagnosing and addressing disparities in machine predictions. However, many machine predictions are deployed to assist in decisions where a human decision-maker retains the ultimate decision authority. In this article, we therefore consider how properties of machine predictions affect the resulting human decisions. We show in a formal model that the inclusion of a biased human decision-maker can revert common relationships between the structure of the algorithm and the qualities of resulting decisions. Specifically, we document that excluding information about protected groups from the prediction may fail to reduce, and may even increase, ultimate disparities. While our concrete results rely on specific assumptions about the data, algorithm, and decision-maker, they show more broadly that any study of critical properties of complex decision systems, such as the fairness of machine-assisted human decisions, should go beyond focusing on the underlying algorithmic predictions in isolation.
翻译:当机器学习算法在高层决策中部署时,我们希望确保机器学习算法的部署能够带来公正和公平的结果。这一关切促使了迅速增长的文献,其重点是诊断和解决机器预测中的差异。然而,许多机器预测用于协助在人类决策者保留最终决策权威时作出决定。因此,在本篇文章中,我们考虑机器预测的特性如何影响人类最终决策。我们在一个正式模型中显示,包含有偏见的人类决策者可以使算法结构与由此产生的决定的质量之间恢复共同的关系。具体地说,我们记录,将受保护群体的信息排除在预测之外可能无法减少甚至可能增加最终差异。虽然我们的具体结果依赖于数据、算法和决策制定者的具体假设,但它们更加广泛地表明,对复杂决策系统的关键属性的任何研究,例如机器辅助人类决策的公正性,应该超越孤立地侧重于基本的算法预测。