Datasets play a central role in the training and evaluation of machine learning (ML) models. But they are also the root cause of many undesired model behaviors, such as biased predictions. To overcome this situation, the ML community is proposing a data-centric cultural shift where data issues are given the attention they deserve, and more standard practices around the gathering and processing of datasets start to be discussed and established. So far, these proposals are mostly high-level guidelines described in natural language and, as such, they are difficult to formalize and apply to particular datasets. In this sense, and inspired by these proposals, we define a new domain-specific language (DSL) to precisely describe machine learning datasets in terms of their structure, data provenance, and social concerns. We believe this DSL will facilitate any ML initiative to leverage and benefit from this data-centric shift in ML (e.g., selecting the most appropriate dataset for a new project or better replicating other ML results). The DSL is implemented as a Visual Studio Code plugin, and it has been published under an open source license.
翻译:数据集在机器学习模式(ML)的培训和评估中发挥着核心作用。 但是,它们也是许多不理想模式行为的根本原因,例如偏向预测。为了克服这种情况,ML社区正在提出以数据为中心的文化转变,使数据问题得到应有的重视,并开始讨论和确立关于收集和处理数据集的更标准的做法。迄今为止,这些建议大多是用自然语言描述的高层次指南,因此难以正式确定和适用于特定的数据集。从这个意义上讲,在这些建议的启发下,我们定义了一种新的特定域语言(DSL),以精确描述机器学习数据集的结构、数据来源和社会关切。我们认为,DLSL将促进任何ML倡议,以便利用和受益于ML的这种以数据为中心的变化(例如,为新项目选择最合适的数据集,或更好地复制其他ML结果)。DSL是作为视觉工作室代码插件实施的,并且根据开放源许可予以公布。