Expert decision-makers (DMs) in high-stakes AI-assisted decision-making (AIaDM) settings receive and reconcile recommendations from AI systems before making their final decisions. We identify distinct properties of these settings which are key to developing AIaDM models that effectively benefit team performance. First, DMs incur reconciliation costs from exerting decision-making resources (e.g., time and effort) when reconciling AI recommendations that contradict their own judgment. Second, DMs in AIaDM settings exhibit algorithm discretion behavior (ADB), i.e., an idiosyncratic tendency to imperfectly accept or reject algorithmic recommendations for any given decision task. The human's reconciliation costs and imperfect discretion behavior introduce the need to develop AI systems which (1) provide recommendations selectively, (2) leverage the human partner's ADB to maximize the team's decision accuracy while regularizing for reconciliation costs, and (3) are inherently interpretable. We refer to the task of developing AI to advise humans in AIaDM settings as learning to advise and we address this task by first introducing the AI-assisted Team (AIaT)-Learning Framework. We instantiate our framework to develop TeamRules (TR): an algorithm that produces rule-based models and recommendations for AIaDM settings. TR is optimized to selectively advise a human and to trade-off reconciliation costs and team accuracy for a given environment by leveraging the human partner's ADB. Evaluations on synthetic and real-world benchmark datasets with a variety of simulated human accuracy and discretion behaviors show that TR robustly improves the team's objective across settings over interpretable, rule-based alternatives.
翻译:高层次的专家决策者(DMs)在作出最终决定之前接受并调和来自AI系统的建议。我们确定这些环境的不同特性,这些特性是开发AIDM模式的关键,这些模型有效地有利于团队业绩。首先,管理模式在调和与其自身判断相矛盾的AI建议时,由于使用决策资源(例如时间和努力),需要支付调和费用。第二,管理模式在AIDM环境中表现出算法自由裁量行为(ADM),即一种不完全接受或拒绝任何特定决策任务的算法行为不精确的典型倾向。人类环境的和解成本和不完善的自由裁量行为使得需要开发AI系统,这些系统(1) 有选择地提供建议,(2) 利用人类伙伴的亚行来尽量提高小组决策的准确性,同时对调和费用进行规范,(3) 本质上是可以解释的。我们提到开发AIDM(ADM)环境中的人的AI)咨询任务,以学习和我们处理这项任务,首先介绍以AI-TR为基础的小组(AIAT)学习的准确性原则框架。我们用一个目标框架来提出一个用于制定规则的精确性仲裁规则,并用ADBDR(ADR)小组来显示一个最精确的仲裁规则的仲裁小组。我们的一个框架,以便提出一套规则和最精确性地评估。我们的一个框架,以提出一个用于研算算出一个标准,以便研算出一个规则的会计规则和最精确性的人权规则,以研算。