Multi-access Edge Computing (MEC) is an emerging computing paradigm that extends cloud computing to the network edge to support resource-intensive applications on mobile devices. As a crucial problem in MEC, service migration needs to decide how to migrate user services for maintaining the Quality-of-Service when users roam between MEC servers with limited coverage and capacity. However, finding an optimal migration policy is intractable due to the dynamic MEC environment and user mobility. Many existing studies make centralized migration decisions based on complete system-level information, which is time-consuming and also lacks desirable scalability. To address these challenges, we propose a novel learning-driven method, which is user-centric and can make effective online migration decisions by utilizing incomplete system-level information. Specifically, the service migration problem is modeled as a Partially Observable Markov Decision Process (POMDP). To solve the POMDP, we design a new encoder network that combines a Long Short-Term Memory (LSTM) and an embedding matrix for effective extraction of hidden information, and further propose a tailored off-policy actor-critic algorithm for efficient training. The extensive experimental results based on real-world mobility traces demonstrate that this new method consistently outperforms both the heuristic and state-of-the-art learning-driven algorithms and can achieve near-optimal results on various MEC scenarios.
翻译:多接入电磁计算(MEC)是一个新兴的计算模式,它将云计算扩展到网络边缘,以支持移动设备的资源密集型应用。作为MEC的一个关键问题,服务性迁移需要决定用户在覆盖和容量有限的MEC服务器之间漫游时如何迁移用户服务,以维持服务质量。然而,由于MEC动态环境和用户流动性,很难找到最佳的移民政策。许多现有研究根据完整的系统级信息做出集中的移民决定,这些信息耗时且缺乏适当的可缩放性。为了应对这些挑战,我们提出了一种新的学习驱动方法,这种方法以用户为中心,能够利用不完整的系统级信息做出有效的在线移民决策。具体地说,服务性迁移问题的模式是部分可观测的Markov 决策程序(POMDP)。为了解决POMDP,我们设计了新的编码网络,将长期短期记忆(LSTM)和有效提取隐藏信息的嵌入式矩阵结合起来,并进一步建议为高效的近距离培训量定制的离政策性行为者-批评性算法,从而能够从现实世界范围内持续学习的各种模式上取得广泛的实验性模型结果。