The quality of Artificial Intelligence (AI) algorithms is of significant importance for confidently adopting algorithms in various applications such as cybersecurity, healthcare, and autonomous driving. This work presents a principled framework of using a design-of-experimental approach to systematically evaluate the quality of AI algorithms, named as Do-AIQ. Specifically, we focus on investigating the quality of the AI mislabel data algorithm against data poisoning. The performance of AI algorithms is affected by hyperparameters in the algorithm and data quality, particularly, data mislabeling, class imbalance, and data types. To evaluate the quality of the AI algorithms and obtain a trustworthy assessment on the quality of the algorithms, we establish a design-of-experiment framework to construct an efficient space-filling design in a high-dimensional constraint space and develop an effective surrogate model using additive Gaussian process to enable the emulation of the quality of AI algorithms. Both theoretical and numerical studies are conducted to justify the merits of the proposed framework. The proposed framework can set an exemplar for AI algorithm to enhance the AI assurance of robustness, reproducibility, and transparency.
翻译:人工智能(AI)算法的质量对于在网络安全、保健和自主驱动等各种应用中自信地采用算法非常重要。这项工作提出了一个原则性框架,即使用实验性设计方法系统评估称为Do-AIQ的人工智能算法的质量。具体地说,我们侧重于调查AI错误标签数据算法与数据中毒的关系的质量。人工智能算法的性能受到算法和数据质量中超参数的影响,特别是数据标签错误、阶级不平衡和数据类型。为了评估AI算法的质量,并获得对算法质量的可靠评估,我们建立了一个设计性实验性框架,以便在高维度制约空间建立一个高效的空间填充设计,并利用添加性高斯进程开发一个有效的代孕模型,以便能够模拟AI算法的质量。进行理论性和数字性研究是为了证明拟议框架的优点。拟议框架可以为AI算法设计一个示例,以加强AI可靠性、可追溯性和透明度的保证。