There are significant benefits to serve deep learning models from relational databases. First, features extracted from databases do not need to be transferred to any decoupled deep learning systems for inferences, and thus the system management overhead can be significantly reduced. Second, in a relational database, data management along the storage hierarchy is fully integrated with query processing, and thus it can continue model serving even if the working set size exceeds the available memory. Applying model deduplication can greatly reduce the storage space, memory footprint, cache misses, and inference latency. However, existing data deduplication techniques are not applicable to the deep learning model serving applications in relational databases. They do not consider the impacts on model inference accuracy as well as the inconsistency between tensor blocks and database pages. This work proposed synergistic storage optimization techniques for duplication detection, page packing, and caching, to enhance database systems for model serving. We implemented the proposed approach in netsDB, an object-oriented relational database. Evaluation results show that our proposed techniques significantly improved the storage efficiency and the model inference latency, and serving models from relational databases outperformed existing deep learning frameworks when the working set size exceeds available memory.
翻译:首先,从数据库中提取的特征不需要转移到任何分解的深层学习系统进行推理,因此系统管理管理管理间接费用可以大大减少。第二,在关系数据库中,存储层的数据管理与查询处理完全结合,因此,即使工作套件的尺寸超过现有内存,也可以继续使用模型。应用模型脱重复可以大大减少存储空间、记忆足迹、缓存漏和推导延迟。然而,现有的数据脱重复技术不适用于为关系数据库应用程序服务的深层学习模型。它们不考虑对模型推导准确性的影响,也不考虑温度区块和数据库网页之间不一致的问题。这项工作提出了协同储存优化技术,用于重复检测、页面包装和缓存,以加强模型使用的数据库系统。我们在以对象为导向的关系数据库中采用了拟议的方法。评价结果表明,我们提议的技术大大改进了存储效率和模型推导延延度,在工作设置时,从关系数据库中获得的模型超过了现有深度学习框架。