With large-scale integration of renewable generation and distributed energy resources (DERs), modern power systems are confronted with new operational challenges, such as growing complexity, increasing uncertainty, and aggravating volatility. Meanwhile, more and more data are becoming available owing to the widespread deployment of smart meters, smart sensors, and upgraded communication networks. As a result, data-driven control techniques, especially reinforcement learning (RL), have attracted surging attention in recent years. In this paper, we provide a tutorial on various RL techniques and how they can be applied to decision-making in power systems. We illustrate RL-based models and solutions in three key applications, frequency regulation, voltage control, and energy management. We conclude with three critical issues in the application of RL, i.e., safety, scalability, and data. Several potential future directions are discussed as well.
翻译:随着可再生能源和分布式能源的大规模整合,现代电力系统面临新的操作挑战,如日益复杂、日益不确定和日益加剧的波动;同时,由于广泛部署智能仪、智能传感器和升级的通信网络,越来越多的数据正在出现;因此,近年来数据驱动控制技术,特别是强化学习,在最近几年引起人们的极大关注;在本文件中,我们就各种RL技术以及如何将其应用于电力系统的决策提供了辅导;我们介绍了基于RL的模型和解决方案在三种关键应用中,即频率调节、电压控制和能源管理。我们最后提出了三个关键问题,即安全、可扩缩性和数据。我们还讨论了若干潜在的未来方向。