Data quality assessment has become a prominent component in the successful execution of complex data-driven artificial intelligence (AI) software systems. In practice, real-world applications generate huge volumes of data at speeds. These data streams require analysis and preprocessing before being permanently stored or used in a learning task. Therefore, significant attention has been paid to the systematic management and construction of high-quality datasets. Nevertheless, managing voluminous and high-velocity data streams is usually performed manually (i.e. offline), making it an impractical strategy in production environments. To address this challenge, DataOps has emerged to achieve life-cycle automation of data processes using DevOps principles. However, determining the data quality based on a fitness scale constitutes a complex task within the framework of DataOps. This paper presents a novel Data Quality Scoring Operations (DQSOps) framework that yields a quality score for production data in DataOps workflows. The framework incorporates two scoring approaches, an ML prediction-based approach that predicts the data quality score and a standard-based approach that periodically produces the ground-truth scores based on assessing several data quality dimensions. We deploy the DQSOps framework in a real-world industrial use case. The results show that DQSOps achieves significant computational speedup rates compared to the conventional approach of data quality scoring while maintaining high prediction performance.
翻译:数据质量评估已成为成功执行复杂数据驱动人工智能(AI)软件系统的重要组成部分。实践中,实际应用程序以极快的速度生成大量的数据流。这些数据流在永久存储或用于学习任务之前需要进行分析和预处理。因此,在构建高质量数据集时,人们已经对系统管理和建设付出了重要的关注。然而,在生产环境中,管理大量和高速度的数据流通常是手动进行的(即线下操作),这使得它成为不可行的策略。为解决这一挑战,DataOps出现了,以使用DevOps原则实现数据过程的生命周期自动化。然而,在DataOps框架内,根据适应度评分来确定数据质量是一个复杂的任务。本文提出了一种新颖的数据质量评分操作(DQSOps)框架,它为DataOps工作流程中的生产数据产生质量分数。该框架结合了两种评分方法,一种基于机器学习预测的方法,用于预测数据质量分数,另一种基于标准的方法,则周期性地根据评估几个数据质量维度产生基础分数。我们在实际工业用例中部署了DQSOps框架,结果表明,相比传统的数据质量评分方法,DQSOps实现了显著的计算加速率,同时保持了高预测性能。