Advancements in ultra-low-power tiny machine learning (TinyML) systems promise to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted and easily reproducible benchmark for these systems. To meet this need, we present MLPerf Tiny, the first industry-standard benchmark suite for ultra-low-power tiny machine learning systems. The benchmark suite is the collaborative effort of more than 50 organizations from industry and academia and reflects the needs of the community. MLPerf Tiny measures the accuracy, latency, and energy of machine learning inference to properly evaluate the tradeoffs between systems. Additionally, MLPerf Tiny implements a modular design that enables benchmark submitters to show the benefits of their product, regardless of where it falls on the ML deployment stack, in a fair and reproducible manner. The suite features four benchmarks: keyword spotting, visual wake words, image classification, and anomaly detection.
翻译:超低能小型机器学习系统(TinyML)的进步有望释放出全新的智能应用类别,然而,由于这些系统缺乏广泛接受和易于复制的基准,持续的进展受到限制。为了满足这一需要,我们提出了超低能小型机器学习系统的第一个工业标准基准套件MLPerf Tini。基准套件是来自工业和学术界的50多个组织的合作努力,反映了社区的需要。MLPerf Tiny 测量了正确评估系统之间交易的机器学习推理的准确性、耐久性和能量。此外,MLPerf Ty 采用模块设计,使基准提交者能够以公平、可复制的方式展示其产品的好处,而不论产品在ML部署堆中的位置如何。套件有四个基准:关键词定位、视觉觉醒词、图像分类和异常检测。