Collaboration among industrial Internet of Things (IoT) devices and edge networks is essential to support computation-intensive deep neural network (DNN) inference services which require low delay and high accuracy. Sampling rate adaption which dynamically configures the sampling rates of industrial IoT devices according to network conditions, is the key in minimizing the service delay. In this paper, we investigate the collaborative DNN inference problem in industrial IoT networks. To capture the channel variation and task arrival randomness, we formulate the problem as a constrained Markov decision process (CMDP). Specifically, sampling rate adaption, inference task offloading and edge computing resource allocation are jointly considered to minimize the average service delay while guaranteeing the long-term accuracy requirements of different inference services. Since CMDP cannot be directly solved by general reinforcement learning (RL) algorithms due to the intractable long-term constraints, we first transform the CMDP into an MDP by leveraging the Lyapunov optimization technique. Then, a deep RL-based algorithm is proposed to solve the MDP. To expedite the training process, an optimization subroutine is embedded in the proposed algorithm to directly obtain the optimal edge computing resource allocation. Extensive simulation results are provided to demonstrate that the proposed RL-based algorithm can significantly reduce the average service delay while preserving long-term inference accuracy with a high probability.
翻译:工业物品互联网(IoT)装置和边缘网络之间的协作对于支持计算密集的深神经网络(DNN)的推断服务至关重要,这些服务需要低延迟和高精确度。抽样率调整,根据网络条件动态地配置工业的IoT装置取样率,是最大限度地减少服务延误的关键。在本文件中,我们调查工业内线网络(IoT)装置和边缘网络中DNN合作的DNN推断问题。为了捕捉频道变异和任务抵达随机性,我们将问题发展成一个有限的Markov决策程序(CMDP)。具体地说,抽样率调整、过载任务和边端计算资源分配被联合考虑,以尽量减少平均服务延迟,同时保证不同推断服务的长期准确性要求。由于总加固学习(RL)算无法直接解决服务延迟的问题,因此我们首先利用Lyapunov优化技术将CMDP转换成一个MDP。然后,提出一个基于深度的RLL算法来解决MDP问题。为了加快培训进程,为了加快培训进程,一个优化的子路段下值分配,然后在提议的高级一级算法中进行最精确的精度分析,然后提出一个最精确的逻辑分析算,以直接显示,以直接获得最优的Rralevalalalalalalalalevalalalalalevalevalalalevaleval。