With smartphones' omnipresence in people's pockets, Machine Learning (ML) on mobile is gaining traction as devices become more powerful. With applications ranging from visual filters to voice assistants, intelligence on mobile comes in many forms and facets. However, Deep Neural Network (DNN) inference remains a compute intensive workload, with devices struggling to support intelligence at the cost of responsiveness.On the one hand, there is significant research on reducing model runtime requirements and supporting deployment on embedded devices. On the other hand, the strive to maximise the accuracy of a task is supported by deeper and wider neural networks, making mobile deployment of state-of-the-art DNNs a moving target. In this paper, we perform the first holistic study of DNN usage in the wild in an attempt to track deployed models and match how these run on widely deployed devices. To this end, we analyse over 16k of the most popular apps in the Google Play Store to characterise their DNN usage and performance across devices of different capabilities, both across tiers and generations. Simultaneously, we measure the models' energy footprint, as a core cost dimension of any mobile deployment. To streamline the process, we have developed gaugeNN, a tool that automates the deployment, measurement and analysis of DNNs on devices, with support for different frameworks and platforms. Results from our experience study paint the landscape of deep learning deployments on smartphones and indicate their popularity across app developers. Furthermore, our study shows the gap between bespoke techniques and real-world deployments and the need for optimised deployment of deep learning models in a highly dynamic and heterogeneous ecosystem.
翻译:随着智能手机在人们口袋中的万无一失,移动上的机器学习(ML)正随着设备变得更加强大而获得牵引力。随着从视觉过滤器到语音助理等各种应用程序的应用,移动的智能以多种形式和方面出现。然而,深神经网络(DNN)的推论仍是一个计算密集的工作量,设备在以反应为代价支持智能方面挣扎着。一方面,在减少模型运行时间需求和支持嵌入设备部署方面进行了大量研究。另一方面,在更深层和更广的神经网络的支持下,努力最大限度地提高任务准确性,使最先进的生态系统技术的移动部署成为移动目标。在本文件中,我们对DNNN在野外的使用情况进行第一次全面研究,试图跟踪部署模式,并匹配这些模式如何在广泛部署的装置上运行。至此,我们分析了谷歌游戏商店中最受欢迎的16k个应用程序,以描述其在不同能力、跨层和代的设备的使用和性能。 同时,我们测量模型的能源足迹,作为动态技术的移动部署的智能模型,以及任何移动工具的深度部署过程的深度分析,我们从一个核心成本的模型的模型和数字的部署过程来展示。