In the traditional vehicular network, computing tasks generated by the vehicles are usually uploaded to the cloud for processing. However, since task offloading toward the cloud will cause a large delay, vehicular edge computing (VEC) is introduced to avoid such a problem and improve the whole system performance, where a roadside unit (RSU) with certain computing capability is used to process the data of vehicles as an edge entity. Owing to the privacy and security issues, vehicles are reluctant to upload local data directly to the RSU, and thus federated learning (FL) becomes a promising technology for some machine learning tasks in VEC, where vehicles only need to upload the local model hyperparameters instead of transferring their local data to the nearby RSU. Furthermore, as vehicles have different local training time due to various sizes of local data and their different computing capabilities, asynchronous federated learning (AFL) is employed to facilitate the RSU to update the global model immediately after receiving a local model to reduce the aggregation delay. However, in AFL of VEC, different vehicles may have different impact on the global model updating because of their various local training delay, transmission delay and local data sizes. Also, if there are bad nodes among the vehicles, it will affect the global aggregation quality at the RSU. To solve the above problem, we shall propose a deep reinforcement learning (DRL) based vehicle selection scheme to improve the accuracy of the global model in AFL of vehicular network. In the scheme, we present the model including the state, action and reward in the DRL based to the specific problem. Simulation results demonstrate our scheme can effectively remove the bad nodes and improve the aggregation accuracy of the global model.
翻译:在传统车辆网络中,由车辆生成的计算任务通常上传到云端进行处理。然而,由于任务向云的卸载会导致较大的延迟,因此引入了车联网边缘计算 (VEC) 来避免这个问题并提高整个系统的性能,其中路边装置 (RSU) 作为边缘实体,具有一定的计算能力用于处理车辆数据。由于隐私和安全问题,车辆不愿直接将本地数据上传到 RSU,因此联邦学习 (FL) 成为 VEC 中某些机器学习任务的一种可行技术,其中车辆只需上传本地模型超参数而不是将其本地数据传输到附近的 RSU。此外,由于车辆具有不同的本地训练时间(由于本地数据大小和计算能力不同),因此采用异步联邦学习 (AFL) 来促进 RSU 在接收到本地模型后立即更新全局模型以减少汇聚延迟。然而,在 VEC 的 AFL 中,不同的车辆可能会对全局模型更新产生不同的影响,因为它们的本地训练延迟,传输延迟和本地数据大小各不相同。此外,如果车辆中有错误节点,将影响 RSU 的全局聚合质量。为解决上述问题,我们提出了一种基于深度强化学习 (DRL) 的车辆选择方案,以提高车联网 AFL 中全局模型的准确性。在该方案中,我们将 DRL 方案的状态、动作和奖励架构具体应用到特定问题中。仿真结果表明,我们的方案可以有效消除错误节点并提高全局模型的聚合准确性。