构建可信赖的自动化：面向初学者的AutoML工具设计原则与评估 (Engineering Trustworthy Automation: Design Principles and Evaluation for AutoML Tools for Novices)

from arxiv, Submitted version accepted for publication in an LNCS Volume "Engineering Interactive Computer Systems - EICS 2025 - International Workshops and Doctoral Consortium"

AutoML systems targeting novices often prioritize algorithmic automation over usability, leaving gaps in users' understanding, trust, and end-to-end workflow support. To address these issues, we propose an abstract pipeline that covers data intake, guided configuration, training, evaluation, and inference. To examine the abstract pipeline, we report a user study where we assess trust, understandability, and UX of a prototype implementation. In a 24-participant study, all participants successfully built their own models, UEQ ratings were positive, yet experienced users reported higher trust and understanding than novices. Based on this study, we propose four design principles to improve the design of AutoML systems targeting novices: (P1) support first-model success to enhance user self-efficacy, (P2) provide explanations to help users form correct mental models and develop appropriate levels of reliance, (P3) provide abstractions and context-aware assistance to keep users in their zone of proximal development, and (P4) ensure predictability and safeguards to strengthen users' sense of control.

翻译：面向初学者的自动机器学习（AutoML）系统往往过于强调算法自动化而忽视了可用性，导致用户在理解、信任及端到端工作流支持方面存在不足。为解决这些问题，我们提出了一个涵盖数据导入、引导式配置、训练、评估与推理的抽象流程框架。为验证该框架，我们通过用户研究评估了原型系统的可信度、可理解性与用户体验。在一项包含24名参与者的研究中，所有参与者均成功构建了各自的模型，用户体验问卷（UEQ）评分总体积极，但有经验的用户比初学者表现出更高的信任度与理解水平。基于此研究，我们提出四项设计原则以改进面向初学者的AutoML系统设计：（P1）保障首次建模成功以提升用户自我效能感；（P2）提供解释机制以帮助用户建立正确心智模型并形成适度依赖；（P3）通过抽象化与情境感知辅助使用户保持在最近发展区内；（P4）确保可预测性与安全防护以增强用户控制感。