Mobile vision systems such as smartphones, drones, and augmented-reality headsets are revolutionizing our lives. These systems usually run multiple applications concurrently and their available resources at runtime are dynamic due to events such as starting new applications, closing existing applications, and application priority changes. In this paper, we present NestDNN, a framework that takes the dynamics of runtime resources into account to enable resource-aware multi-tenant on-device deep learning for mobile vision systems. NestDNN enables each deep learning model to offer flexible resource-accuracy trade-offs. At runtime, it dynamically selects the optimal resource-accuracy trade-off for each deep learning model to fit the model's resource demand to the system's available runtime resources. In doing so, NestDNN efficiently utilizes the limited resources in mobile vision systems to jointly maximize the performance of all the concurrently running applications. Our experiments show that compared to the resource-agnostic status quo approach, NestDNN achieves as much as 4.2% increase in inference accuracy, 2.0x increase in video frame processing rate and 1.7x reduction on energy consumption.
翻译:智能手机、无人驾驶飞机和高级现实耳机等移动视觉系统正在使我们的生活发生革命性的变化。 这些系统通常同时运行多个应用程序,而且由于启动新应用程序、关闭现有应用程序和应用优先性变化等事件,它们可用的资源在运行时是动态的。 在本文中,我们介绍了NestDNN, 这个框架将运行时资源的动态考虑在内,使拥有资源认知的多租租户能够深入深造移动视觉系统。 NestDNN使每个深层次学习模式能够提供灵活的资源准确性交换。 在运行时,它动态地为每个深层学习模型选择最佳的资源准确性交换,以适应该模型的资源需求与系统现有的运行时间资源的需求。 在这样做时,NestDNNN有效地利用移动视觉系统中的有限资源,共同最大限度地提高同时运行的所有应用程序的性能。 我们的实验表明,与资源可靠现状方法相比,NestDNNNNNNN能够达到4.2%的推算准确性增长、2.0x的视频框架处理率和1.7x能源消耗减少率。