Risk Assessment Instruments (RAIs) are widely used to forecast adverse outcomes in domains such as healthcare and criminal justice. RAIs are commonly trained on observational data and are optimized to predict observable outcomes rather than potential outcomes, which are the outcomes that would occur absent a particular intervention. Examples of relevant potential outcomes include whether a patient's condition would worsen without treatment or whether a defendant would recidivate if released pretrial. We illustrate how RAIs which are trained to predict observable outcomes can lead to worse decision making, causing precisely the types of harm they are intended to prevent. This can occur even when the predictors are Bayes-optimal and there is no unmeasured confounding.
翻译:风险评估工具(RAI)被广泛用于预测保健和刑事司法等领域的不利结果,RAIs通常接受观察数据培训,并被优化以预测可观测结果,而不是潜在结果,这些结果是在没有特定干预的情况下会出现的结果,有关潜在结果的例子包括:患者的病情是否在没有治疗的情况下会恶化,或者被告是否在审前释放后会复发。我们说明受过培训以预测可观察结果的RAIs如何会导致更糟糕的决策,从而造成他们打算预防的伤害的准确类型。即使预测器是最佳的,而且没有不完全的混杂,这种情况也可能发生。