Recent work shows the effectiveness of Machine Learning (ML) to reduce cache miss ratios by making better eviction decisions than heuristics. However, state-of-the-art ML caches require many predictions to make an eviction decision, making them impractical for high-throughput caching systems. This paper introduces Machine learning At the Tail (MAT), a framework to build efficient ML-based caching systems by integrating an ML module with a traditional cache system based on a heuristic algorithm. MAT treats the heuristic algorithm as a filter to receive high-quality samples to train an ML model and likely candidate objects for evictions. We evaluate MAT on 8 production workloads, spanning storage, in-memory caching, and CDNs. The simulation experiments show MAT reduces the number of costly ML predictions-per-eviction from 63 to 2, while achieving comparable miss ratios to the state-of-the-art ML cache system. We compare a MAT prototype system with an LRU-based caching system in the same setting and show that they achieve similar request rates.
翻译:最近的工作表明,机器学习(ML)通过作出更好的驱逐决定来减少缓存误差比率,比超自然学更能有效减少缓存误差比率。然而,最先进的ML缓存需要许多预测才能作出驱逐决定,使高通量缓冲系统不切实际。本文介绍了机器学习在尾部(MAT)这一框架,通过将一个ML模块与基于超自然算法的传统缓存系统整合起来来建立高效的ML缓存系统。MAT将超自然算法当作接受高质量样本的过滤器,以训练ML模型和可能的驱逐候选对象。我们评估了8个生产工作量的MAT,横跨存储、模拟缓存和CDN。模拟实验显示,MAT将昂贵的ML预测- Per-evection数量从63降至2,同时实现与最先进的ML的缓存系统相似的误差比率。我们将一个MAT原型系统与同一环境中的基于LU的缓存系统进行比较,并显示它们达到类似的请求率。