Despite an increasing reliance on fully-automated algorithmic decision-making in our day-to-day lives, human beings still make highly consequential decisions. As frequently seen in business, healthcare, and public policy, recommendations produced by algorithms are provided to human decision-makers to guide their decisions. While there exists a fast-growing literature evaluating the bias and fairness of such algorithmic recommendations, an overlooked question is whether they help humans make better decisions. We develop a statistical methodology for experimentally evaluating the causal impacts of algorithmic recommendations on human decisions. We also show how to examine whether algorithmic recommendations improve the fairness of human decisions and derive the optimal decision rules under various settings. We apply the proposed methodology to preliminary data from the first-ever randomized controlled trial that evaluates the pretrial Public Safety Assessment (PSA) in the criminal justice system. A goal of the PSA is to help judges decide which arrested individuals should be released. On the basis of the preliminary data available, we find that providing the PSA to the judge has little overall impact on the judge's decisions and subsequent arrestee behavior. However, our analysis yields some potentially suggestive evidence that the PSA may help avoid unnecessarily harsh decisions for female arrestees regardless of their risk levels while it encourages the judge to make stricter decisions for male arrestees who are deemed to be risky. In terms of fairness, the PSA appears to increase the gender bias against males while having little effect on any existing racial differences in judges' decision. Finally, we find that the PSA's recommendations might be unnecessarily severe unless the cost of a new crime is sufficiently high.
翻译:尽管在日常生活中日益依赖完全自动化的算法决策,但人仍然会作出非常重大的决定。正如商业、保健和公共政策中经常看到的那样,算法所产生的建议会提供给人类决策者,以指导他们的决定。虽然评估这种算法建议的偏差和公正性的文献迅速增加,但一个被忽视的问题是,他们是否帮助人类作出更好的决定。我们开发了一种统计方法,用于实验性地评价算法建议对人类决定的因果影响。我们还展示了如何审查算法建议是否提高了人类决定的公平性,并在各种情况下得出了最佳的决策规则。我们运用了拟议方法,用于初步数据,用于评估刑事司法系统中首次随机控制的受控审判,以指导其决定。虽然对审前公共安全评估(PSA)的评估是一个快速增长的文献,目的是帮助法官决定谁被捕。根据现有的初步数据,我们发现向法官提供PSA的少量犯罪对法官的决定和随后被捕者的行为没有多少全面影响。然而,我们的分析提供了一些有潜在严重证据,表明PSA的公正性可能避免他们作出更严厉的决定,而法官最终却认为PSA的性别风险程度会充分鼓励法官作出更严厉的决定。