In reaction to growing concerns about the potential harms of artificial intelligence (AI), societies have begun to demand more transparency about how AI models and systems are created and used. To address these concerns, several efforts have proposed documentation templates containing questions to be answered by model developers. These templates provide a useful starting point, but no single template can cover the needs of diverse documentation consumers. It is possible in principle, however, to create a repeatable methodology to generate truly useful documentation. Richards et al. [25] proposed such a methodology for identifying specific documentation needs and creating templates to address those needs. Although this is a promising proposal, it has not been evaluated. This paper presents the first evaluation of this user-centered methodology in practice, reporting on the experiences of a team in the domain of AI for healthcare that adopted it to increase transparency for several AI models. The methodology was found to be usable by developers not trained in user-centered techniques, guiding them to creating a documentation template that addressed the specific needs of their consumers while still being reusable across different models and use cases. Analysis of the benefits and costs of this methodology are reviewed and suggestions for further improvement in both the methodology and supporting tools are summarized.
翻译:由于人们对人工智能的潜在危害日益感到关切(AI),社会已开始要求更加透明地了解如何创建和使用人工智能模式和系统;为解决这些关切,作出了几项努力,提出了文件模板,其中载有有待示范开发者回答的问题,这些模板提供了一个有益的起点,但没有一个单一模板能够满足不同文件消费者的需要,然而,原则上可以制定可重复使用的方法,以产生真正有用的文件。Richards等人[25] 提出了这样一个方法,用以确定具体的文件需要,并为满足这些需要创建模板。虽然这是一个很有希望的建议,但该文件没有被评估。本文件介绍了对这个以用户为中心的方法的第一次评估,并报告了大赦国际保健领域一个小组的经验,该小组采纳了这些经验,以提高若干人工智能模式的透明度。该方法被未受过用户中心技术培训的开发者使用,指导他们创建一个满足其消费者具体需要的文件模板,同时在不同模式和使用案例中仍然可以重新使用。对这种方法的效益和成本进行了分析,并总结了该方法的效益和成本,同时总结了方法和辅助工具的进一步改进建议。