Nowadays, many AI applications utilizing resource-constrained edge devices (e.g., small mobile robots, tiny IoT devices, etc.) require Convolutional Neural Network (CNN) inference on a distributed system at the edge due to limited resources of a single edge device to accommodate and execute a large CNN. There are four main partitioning strategies that can be utilized to partition a large CNN model and perform distributed CNN inference on multiple devices at the edge. However, to the best of our knowledge, no research has been conducted to investigate how these four partitioning strategies affect the energy consumption per edge device. Such an investigation is important because it will reveal the potential of these partitioning strategies to be used effectively for reduction of the per-device energy consumption when a large CNN model is deployed for distributed inference at the edge. Therefore, in this paper, we investigate and compare the per-device energy consumption of CNN model inference at the edge on a distributed system when the four partitioning strategies are utilized. The goal of our investigation and comparison is to find out which partitioning strategies (and under what conditions) have the highest potential to decrease the energy consumption per edge device when CNN inference is performed at the edge on a distributed system.
翻译:目前,许多使用资源限制边缘装置(如小型移动机器人、小型IOT装置等)的AI应用软件都需要在边缘对分布式系统进行进化神经网络(CNN)的推断,因为单一边缘装置的资源有限,无法容纳和执行大型CNN。有四种主要分隔战略可用于分割大型CNN模型,并在边缘对多个装置进行分布式CNN推断。然而,根据我们所知,没有开展任何研究来调查这四种分割战略如何影响每个边缘装置的能源消耗。这种调查很重要,因为一旦部署大型CNN模型用于在边缘进行分布式推断时,将揭示这些分割式战略有效用于减少每个设备耗能消耗的潜力。因此,在本文件中,我们调查并比较CNN模型在使用四个隔离战略时对分布式系统边缘的渗透性能源消耗量。我们调查与比较的目的是查明在哪些分区战略(和在什么条件下)在哪些条件下,这些战略能够有效用于减少每个设备耗能消耗量。当CNN系统进行最有可能降低能源消耗的边缘时,我们调查并比较CNN系统在什么条件下进行最有可能降低其分布式边缘。