We are witnessing the emergence of an AI economy and society where AI technologies are increasingly impacting health care, business, transportation and many aspects of everyday life. Many successes have been reported where AI systems even surpassed the accuracy of human experts. However, AI systems may produce errors, can exhibit bias, may be sensitive to noise in the data, and often lack technical and judicial transparency resulting in reduction in trust and challenges in their adoption. These recent shortcomings and concerns have been documented in scientific but also in general press such as accidents with self driving cars, biases in healthcare, hiring and face recognition systems for people of color, seemingly correct medical decisions later found to be made due to wrong reasons etc. This resulted in emergence of many government and regulatory initiatives requiring trustworthy and ethical AI to provide accuracy and robustness, some form of explainability, human control and oversight, elimination of bias, judicial transparency and safety. The challenges in delivery of trustworthy AI systems motivated intense research on explainable AI systems (XAI). Aim of XAI is to provide human understandable information of how AI systems make their decisions. In this paper we first briefly summarize current XAI work and then challenge the recent arguments of accuracy vs. explainability for being mutually exclusive and being focused only on deep learning. We then present our recommendations for the use of XAI in full lifecycle of high stakes trustworthy AI systems delivery, e.g. development, validation and certification, and trustworthy production and maintenance.
翻译:我们目睹了AI经济和社会的出现,AI技术对保健、商业、交通和日常生活的许多方面产生了越来越大的影响,据报告,在AI系统甚至超过人类专家的准确性的地方,AI系统取得了许多成功;然而,AI系统可能产生错误,可能表现出偏差,可能对数据的噪音敏感,而且往往缺乏技术和司法透明度,从而减少了信任和采用这些系统的挑战。这些最近的缺陷和关切在科学和一般报刊上都有记载,例如自驾汽车事故、保健、雇用和对有色人的承认系统方面的偏见、保健、雇用和面部识别系统等,后来发现由于错误的原因等原因作出似乎正确的医疗决定。这导致出现了许多政府和监管举措,需要可信和道德的AI提供准确性和可靠性、某种形式的解释性、人的控制和监督、消除偏见、司法透明度和安全。由于在提供可靠的AI系统方面的挑战,对可解释的AI系统(XAI)进行了深入的研究。 XAI的目的是提供人能理解的关于AI系统如何作出决定的信息。 在这份文件中,我们首先简要地总结了XAI目前的工作,然后对最近关于准确性和可靠性的论点提出了质疑。