To design with AI models, user experience (UX) designers must assess the fit between the model and user needs. Based on user research, they need to contextualize the model's behavior and potential failures within their product-specific data instances and user scenarios. However, our formative interviews with ten UX professionals revealed that such a proactive discovery of model limitations is challenging and time-intensive. Furthermore, designers often lack technical knowledge of AI and accessible exploration tools, which challenges their understanding of model capabilities and limitations. In this work, we introduced a failure-driven design approach to AI, a workflow that encourages designers to explore model behavior and failure patterns early in the design process. The implementation of fAIlureNotes, a designer-centered failure exploration and analysis tool, supports designers in evaluating models and identifying failures across diverse user groups and scenarios. Our evaluation with UX practitioners shows that fAIlureNotes outperforms today's interactive model cards in assessing context-specific model performance.
翻译:在设计AI模型时,用户经验(UX)设计师必须评估模型与用户需求之间的适当性。根据用户研究,他们需要将模型的行为和潜在失败情况与其产品特定数据实例和用户假设情景联系起来。然而,我们与10名UX专业人员的成型访谈表明,这种主动发现模型局限性的做法具有挑战性和时间密集性。此外,设计师往往缺乏对AI的技术知识和无障碍探索工具,这对其理解模型能力和限制提出了挑战。在这项工作中,我们引入了一种由失败驱动的设计方法。在设计过程中,我们引入了一种鼓励设计师在设计过程中早期探索模型行为和失败模式的工作流程。FAIlureNotes是一个以设计师为中心的失败探索和分析工具,它支持设计师评估模型,并查明不同用户群体和情景的失败。我们与UX执行人员进行的评估表明,FAIureNotes在评估特定背景模型性能时,超越了今天的互动模型卡。