Opportunistic computation offloading is an effective method to improve the computation performance of mobile-edge computing (MEC) networks under dynamic edge environment. In this paper, we consider a multi-user MEC network with time-varying wireless channels and stochastic user task data arrivals in sequential time frames. In particular, we aim to design an online computation offloading algorithm to maximize the network data processing capability subject to the long-term data queue stability and average power constraints. The online algorithm is practical in the sense that the decisions for each time frame are made without the assumption of knowing future channel conditions and data arrivals. We formulate the problem as a multi-stage stochastic mixed integer non-linear programming (MINLP) problem that jointly determines the binary offloading (each user computes the task either locally or at the edge server) and system resource allocation decisions in sequential time frames. To address the coupling in the decisions of different time frames, we propose a novel framework, named LyDROO, that combines the advantages of Lyapunov optimization and deep reinforcement learning (DRL). Specifically, LyDROO first applies Lyapunov optimization to decouple the multi-stage stochastic MINLP into deterministic per-frame MINLP subproblems. By doing so, it guarantees to satisfy all the long-term constraints by solving the per-frame subproblems that are much smaller in size. Then, LyDROO integrates model-based optimization and model-free DRL to solve the per-frame MINLP problems with low computational complexity. Simulation results show that under various network setups, the proposed LyDROO achieves optimal computation performance while stabilizing all queues in the system. Besides, it induces very low execution latency that is particularly suitable for real-time implementation in fast fading environments.
翻译:机会性计算卸载是一个有效的方法, 可以在动态边缘环境中改进移动- 高级计算( MEC) 网络的计算性能。 在本文中, 我们考虑建立一个多用户 MEC 网络, 使用时间变化的无线频道和随机用户任务数据在顺序时间框架内到达。 特别是, 我们的目标是设计一个在线计算卸载算法, 在长期数据列队稳定性和平均功率限制的前提下, 最大限度地扩大网络数据处理能力。 在线算法是实用的, 因为每个时间框架的计算都假设了解未来频道条件和数据到达, 而每个时间框架的计算都是在不理解未来频道条件和数据到达的情况下进行。 我们把问题发展成多阶段性平级的混合整流型非线性调整( MINLP P) 混合非线性编程程序( MINLPL Pl), 共同决定双向双向双向双向双向的双向运行运行, 将所有运行中最优性平流的运行系统 向下进行最优的运行运行。 我们提议一个更小型的模拟的系统, 将最低性平级的运行运行运行运行运行的运行在每平级的运行的运行中, 运行的运行的运行的运行中, 向下进行最优化的运行的运行的运行的运行的运行的运行中, 向下进行最优化的运行的运行的运行的运行的运行中, 向下运行的运行的运行的运行的运行中, 向的运行的运行的运行的运行中, 向的运行的运行到最后端的运行的运行式的运行式的运行的运行的运行式的运行式的运行式的运行式的运行式的运行式的运行。