The U.S. criminal legal system increasingly relies on software output to convict and incarcerate people. In a large number of cases each year, the government makes these consequential decisions based on evidence from statistical software -- such as probabilistic genotyping, environmental audio detection, and toolmark analysis tools -- that defense counsel cannot fully cross-examine or scrutinize. This undermines the commitments of the adversarial criminal legal system, which relies on the defense's ability to probe and test the prosecution's case to safeguard individual rights. Responding to this need to adversarially scrutinize output from such software, we propose robust adversarial testing as an audit framework to examine the validity of evidentiary statistical software. We define and operationalize this notion of robust adversarial testing for defense use by drawing on a large body of recent work in robust machine learning and algorithmic fairness. We demonstrate how this framework both standardizes the process for scrutinizing such tools and empowers defense lawyers to examine their validity for instances most relevant to the case at hand. We further discuss existing structural and institutional challenges within the U.S. criminal legal system that may create barriers for implementing this and other such audit frameworks and close with a discussion on policy changes that could help address these concerns.
翻译:美国刑事法律制度日益依赖软件产出来定罪和监禁人。在每年大量案件中,政府根据统计软件的证据做出这些相应的决定 -- -- 例如概率性基因组学、环境音频探测和工具标记分析工具 -- -- 辩护律师无法充分交叉检查或仔细检查。这破坏了对抗性刑事法律制度的承诺,而对抗性刑事法律制度依赖于辩方调查和测试起诉案件的能力,以保障个人权利。针对这种需要,我们提议进行强有力的对抗性测试,作为审查证据性统计软件有效性的审计框架。我们通过借鉴大量近期在强有力的机器学习和算法公正方面开展的工作,界定并落实这一防御性激烈对抗性测试概念。我们证明这一框架如何使审查这类工具的程序标准化,并授权辩护律师审查这些工具的有效性,以了解与本案最相关的案例。我们进一步讨论美国刑事法律制度中现有的结构性和体制性挑战,以审查证据性统计软件的有效性。我们通过界定并运用这一概念,利用大量近期在强有力的机器学习和算法公正方面开展的工作,来界定和运用这种防御性对抗性测试来进行辩护性测试。我们证明这一框架是如何使这类工具的精细化过程标准化,并授权辩护律师对与本案最为相关的案例进行审查。我们进一步讨论。我们进一步讨论刑事法律制度中可能妨碍执行这一框架和其他审计框架,并接近于讨论这些政策。