Recent years have witnessed an interesting phenomenon in which users come together to interrogate potentially harmful algorithmic behaviors they encounter in their everyday lives. Researchers have started to develop theoretical and empirical understandings of these user driven audits, with a hope to harness the power of users in detecting harmful machine behaviors. However, little is known about user participation and their division of labor in these audits, which are essential to support these collective efforts in the future. Through collecting and analyzing 17,984 tweets from four recent cases of user driven audits, we shed light on patterns of user participation and engagement, especially with the top contributors in each case. We also identified the various roles user generated content played in these audits, including hypothesizing, data collection, amplification, contextualization, and escalation. We discuss implications for designing tools to support user driven audits and users who labor to raise awareness of algorithm bias.
翻译:近年来,我们见证了一种有趣的现象,即用户聚集在一起,询问他们在日常生活中遇到的潜在有害算法行为。研究人员已开始发展理论和实证理解这些用户驱动的审计,希望利用用户的力量来检测有害机器行为。然而,我们很少了解用户在这些审计中的参与和劳动分工,这对于未来支持这些集体努力是至关重要的。通过收集和分析来自四个最近用户驱动审计案例的17,984条推文,我们揭示了用户参与和参与度的模式,特别是在每个案例的最高贡献者中。我们还确定了用户生成内容在这些审计中发挥的各种角色,包括假设、数据收集、放大、情境化和升级。我们讨论了为设计支持用户驱动审计的工具和为提高对算法偏见的意识而劳动的用户提供帮助的意义。