The rise of big data has revolutionized data exploitation practices and led to the emergence of new concepts. Among them, data lakes have emerged as large heterogeneous data repositories that can be analyzed by various methods. An efficient data lake requires a metadata system that addresses the many problems arising when dealing with big data. In consequence, the study of data lake metadata models is currently an active research topic and many proposals have been made in this regard. However, existing metadata models are either tailored for a specific use case or insufficiently generic to manage different types of data lakes, including our previous model MEDAL. In this paper, we generalize MEDAL's concepts in a new metadata model called goldMEDAL. Moreover, we compare goldMEDAL with the most recent state-of-the-art metadata models aiming at genericity and show that we can reproduce these metadata models with goldMEDAL's concepts. As a proof of concept, we also illustrate that goldMEDAL allows the design of various data lakes by presenting three different use cases.
翻译:海量数据的崛起使数据开发做法发生了革命性的变化,并导致出现了新的概念,其中包括,数据湖已经作为大型的多样化数据储存库出现,可以采用各种方法进行分析。高效的数据湖需要一个元数据系统,解决在处理海量数据时出现的许多问题。因此,研究数据湖元数据模型目前是一个积极的研究课题,在这方面提出了许多建议。然而,现有的元数据模型不是针对特定用途案例而定制,就是不够通用,无法管理不同种类的数据湖,包括我们以前的模型MEDAL。在本文中,我们把MEDAL的概念概括成一个新的元数据模型,称为GoldmedAL。此外,我们把GoldMEDAL的概念与最新最先进的、旨在通用的元数据模型进行比较,并表明我们可以用GoldMEDAL的概念复制这些元数据模型。作为概念的证明,我们还说明GoldMEDAL允许通过提出三个不同的使用案例来设计不同的数据湖。