The speed and scale at which machine learning (ML) systems are deployed are accelerating even as an increasing number of studies highlight their potential for negative impact. There is a clear need for companies and regulators to manage the risk from proposed ML systems before they harm people. To achieve this, private and public sector actors first need to identify the risks posed by a proposed ML system. A system's overall risk is influenced by its direct and indirect effects. However, existing frameworks for ML risk/impact assessment often address an abstract notion of risk or do not concretize this dependence. We propose to address this gap with a context-sensitive framework for identifying ML system risks comprising two components: a taxonomy of the first- and second-order risks posed by ML systems, and their contributing factors. First-order risks stem from aspects of the ML system, while second-order risks stem from the consequences of first-order risks. These consequences are system failures that result from design and development choices. We explore how different risks may manifest in various types of ML systems, the factors that affect each risk, and how first-order risks may lead to second-order effects when the system interacts with the real world. Throughout the paper, we show how real events and prior research fit into our Machine Learning System Risk framework (MLSR). MLSR operates on ML systems rather than technologies or domains, recognizing that a system's design, implementation, and use case all contribute to its risk. In doing so, it unifies the risks that are commonly discussed in the ethical AI community (e.g., ethical/human rights risks) with system-level risks (e.g., application, design, control risks), paving the way for holistic risk assessments of ML systems.
翻译:安装机器学习(ML)系统的速度和规模正在加速,即使越来越多的研究强调其潜在负面影响,也正在加速部署机器学习(ML)系统的速度和规模。公司和监管者显然需要在伤害人之前管理拟议的ML系统的风险。为此,私营和公共部门行为者首先需要查明拟议的ML系统构成的风险。一个系统的总体风险受其直接和间接影响。但是,现有的ML风险/影响评估框架往往涉及一种抽象的风险概念,或者没有具体体现这种依赖性。我们提议用一个对背景敏感的框架来弥补这一差距,以确定ML系统的风险,包括两个组成部分:ML系统构成的第一和第二级风险的分类,以及其促成因素。要实现这一点,一阶风险来自拟议的ML系统的各个方面,而二阶风险则来自一阶风险的后果。这些后果是设计和发展选择造成的系统失灵。我们探索在各种ML系统、影响每项风险的因素以及非一阶风险如何导致ML系统产生第二阶级风险。当我们实际设计风险时,ML系统如何适应实际设计风险,而ML系统又如何适应了我们实际的ML系统。