With the rapid increasing of data scale, in-database analytics and learning has become one of the most studied topics in data science community, because of its significance on reducing the gap between the management and the analytics of data. By extending the capability of database on analytics and learning, data scientists can save much time on exchanging data between databases and external analytic tools. For this goal, researchers are attempting to integrate more data science algorithms into database. However, implementing the algorithms in mainstream databases is super time-consuming, especially when it is necessary to have a deep dive into the database kernels. Thus there are demands for an easy-to-extend database simulator to help fast prototype and verify the in-database algorithms before implementing them in real databases. In this demo, we present such an extensible relational database simulator, DBSim, to help data scientists prototype their in-database analytics and learning algorithms and verify the effectiveness of their ideas with minimal cost. DBSim simulates a real relational database by integrating all the major components in mainstream RDBMS, including SQL parser, relational operators, query optimizer, etc. In addition, DBSim provides various interfaces for users to flexibly plug their custom extension modules into any of the major components, without modifying the kernel. By those interfaces, DBSim supports easy extensions on SQL syntax, relational operators, query optimizer rules and cost models, and physical plan execution. Furthermore, DBSim provides utilities to facilitate users' developing and debugging, like query plan visualizer and interactive analyzer on optimization rules. We develop DBSim using pure Python to support seamless implementation of most data science algorithms into it, since many of them are written in Python.
翻译:随着数据规模的迅速扩大,数据库内的分析和学习已成为数据科学界中研究最多的课题之一,因为其重要性在于缩小数据管理与分析之间的差距。通过扩大分析与学习数据库的能力,数据科学家可以在数据库与外部分析工具之间交换数据方面节省大量时间。为此,研究人员正在试图将更多的数据科学算法纳入数据库。然而,在主流数据库中应用算法非常耗时,特别是当需要深入潜入数据库内时,更需要将优化算法纳入数据库内层。因此,需要建立一个方便到扩展的数据库模拟器,以帮助快速原型和校验数据库内算法,然后在实际数据库中应用。在此演示中,我们展示了这样一个可扩展的关系数据库模拟器,DBS,在数据库内将大部分分析与学习算法进行原型,并用极低的成本来核查其想法的效用。DBS,通过将真实的直流规则与主机机机内的所有主机内结构进行模拟关系。