Concerns about the societal impact of AI-based services and systems has encouraged governments and other organisations around the world to propose AI policy frameworks to address fairness, accountability, transparency and related topics. To achieve the objectives of these frameworks, the data and software engineers who build machine-learning systems require knowledge about a variety of relevant supporting tools and techniques. In this paper we provide an overview of technologies that support building trustworthy machine learning systems, i.e., systems whose properties justify that people place trust in them. We argue that four categories of system properties are instrumental in achieving the policy objectives, namely fairness, explainability, auditability and safety & security (FEAS). We discuss how these properties need to be considered across all stages of the machine learning life cycle, from data collection through run-time model inference. As a consequence, we survey in this paper the main technologies with respect to all four of the FEAS properties, for data-centric as well as model-centric stages of the machine learning system life cycle. We conclude with an identification of open research problems, with a particular focus on the connection between trustworthy machine learning technologies and their implications for individuals and society.
翻译:为了实现这些框架的目标,建立机器学习系统的数据和软件工程师需要了解各种相关的辅助工具和技术。在本文件中,我们概述了支持建立可靠的机器学习系统的技术,即其性质证明人们信任机器的系统。我们争辩说,四类系统特性对实现政策目标至关重要,即公平、可解释性、可审计性和安全及保障。我们讨论如何在机器学习生命周期的所有阶段,从数据收集到运行时间模型推算,考虑这些特性。因此,我们在本文件中调查FEAS所有四个特性的主要技术,即以数据为中心的以及机器学习系统生命周期以模型为中心的阶段。我们最后指出公开研究问题,特别侧重于可信赖的机器学习技术与对个人和社会的影响之间的联系。