Recent years have seen the development of many open-source ML fairness toolkits aimed at helping ML practitioners assess and address unfairness in their systems. However, there has been little research investigating how ML practitioners actually use these toolkits in practice. In this paper, we conducted the first in-depth empirical exploration of how industry practitioners (try to) work with existing fairness toolkits. In particular, we conducted think-aloud interviews to understand how participants learn about and use fairness toolkits, and explored the generality of our findings through an anonymous online survey. We identified several opportunities for fairness toolkits to better address practitioner needs and scaffold them in using toolkits effectively and responsibly. Based on these findings, we highlight implications for the design of future open-source fairness toolkits that can support practitioners in better contextualizing, communicating, and collaborating around ML fairness efforts.
翻译:近些年来,我们开发了许多开放源码的ML公平工具包,旨在帮助ML从业者评估和解决其系统中的不公平现象,然而,几乎没有研究调查ML从业者如何实际使用这些工具包,在本文件中,我们首次深入地进行了实证探索,探讨行业从业者(努力)如何利用现有的公平工具包开展工作,特别是,我们进行了深思熟虑的访谈,以了解参与者如何学习和使用公平工具包,并通过匿名在线调查探索了我们调查结果的普遍性。我们找到了公平工具包的若干机会,以更好地满足从业者的需求,并在有效和负责任地使用工具包时对他们进行筛选。基于这些发现,我们强调今后设计开放源公平工具包的影响,这些工具包可以支持从业者更好地认识背景、沟通和围绕ML公平努力开展协作。