In spite of machine learning's rapid growth, its engineering support is scattered in many forms, and tends to favor certain engineering stages, stakeholders, and evaluation preferences. We envision a capability-based framework, which uses fine-grained specifications for ML model behaviors to unite existing efforts towards better ML engineering. We use concrete scenarios (model design, debugging, and maintenance) to articulate capabilities' broad applications across various different dimensions, and their impact on building safer, more generalizable and more trustworthy models that reflect human needs. Through preliminary experiments, we show capabilities' potential for reflecting model generalizability, which can provide guidance for ML engineering process. We discuss challenges and opportunities for capabilities' integration into ML engineering.
翻译:尽管机器学习迅速发展,但其工程支持以多种形式分散,倾向于偏好某些工程阶段、利益攸关方和评价偏好。我们设想了一个基于能力的框架,用细细的规格来说明ML模型行为,将现有的努力结合起来,改进ML工程。我们使用具体的设想(模型设计、调试和维护)来说明能力在不同层面的广泛应用,及其对建设更安全、更普遍和更可信赖、反映人类需要的模型的影响。我们通过初步实验,展示了反映模型通用性的潜力,为ML工程进程提供指导。我们讨论了将能力纳入ML工程的挑战和机会。