Wireless networked control system (WNCS) connecting sensors, controllers, and actuators via wireless communications is a key enabling technology for highly scalable and low-cost deployment of control systems in the Industry 4.0 era. Despite the tight interaction of control and communications in WNCSs, most existing works adopt separative design approaches. This is mainly because the co-design of control-communication policies requires large and hybrid state and action spaces, making the optimal problem mathematically intractable and difficult to be solved effectively by classic algorithms. In this paper, we systematically investigate deep learning (DL)-based estimator-control-scheduler co-design for a model-unknown nonlinear WNCS over wireless fading channels. In particular, we propose a co-design framework with the awareness of the sensor's age-of-information (AoI) states and dynamic channel states. We propose a novel deep reinforcement learning (DRL)-based algorithm for controller and scheduler optimization utilizing both model-free and model-based data. An AoI-based importance sampling algorithm that takes into account the data accuracy is proposed for enhancing learning efficiency. We also develop novel schemes for enhancing the stability of joint training. Extensive experiments demonstrate that the proposed joint training algorithm can effectively solve the estimation-control-scheduling co-design problem in various scenarios and provide significant performance gain compared to separative design and some benchmark policies.
翻译:无线网络控制系统(WNCS)连接传感器、控制器和通过无线通信的电动装置,通过无线通信连接传感器、控制器和促动器,是工业4.0时代高可扩缩和低成本部署控制系统的关键赋能技术。尽管在无线通信系统的控制和通信方面相互作用密切,但大多数现有工程采用了分离的设计方法。这主要是因为联合设计控制通信政策需要庞大和混合的状态和行动空间,使得最优问题在数学上难以解决,难以通过经典算法得到有效解决。在本文中,我们系统调查了深层次学习(DL)基于天顶点控制-控制调度器的共同设计,以便在无线淡化的频道上设计出一个不为人知的非线性 WNCNS系统。特别是我们提出了一个共同设计框架,意识到传感器的年龄信息(AoI)状态和动态频道状态。我们提议利用无模式和基于模型的数据,为控制器和定线优化提供新的强化学习(DRL)的算法。基于AOI的重要取样算法,该算法考虑到无线的模型的模型非线性非线定式非线式非线式非线定式非线式非线定序。我们还参照的模型设计中的一些重要测算法,有效地计算方法,以考虑了数据设计模型的模型的模型设计设计模型的模型的模型的模型设计模型设计,可以有效地展示模型设计模型设计,从而展示数据,用以测量测算法。我们为改进式的模型的模型的模型,为改进式的模型的模型的模型的模型,可以展示式测算法。我们用测算法。我们为改进式的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的演算法,可以展示的演算法,可以展示的演算法,可以展示的演算法,可以提高的演算法,可以提高的模型的