Driven by the global decarbonization effort, the rapid integration of renewable energy into the conventional electricity grid presents new challenges and opportunities for the battery energy storage system (BESS) participating in the energy market. Energy arbitrage can be a significant source of revenue for the BESS due to the increasing price volatility in the spot market caused by the mismatch between renewable generation and electricity demand. In addition, the Frequency Control Ancillary Services (FCAS) markets established to stabilize the grid can offer higher returns for the BESS due to their capability to respond within milliseconds. Therefore, it is crucial for the BESS to carefully decide how much capacity to assign to each market to maximize the total profit under uncertain market conditions. This paper formulates the bidding problem of the BESS as a Markov Decision Process, which enables the BESS to participate in both the spot market and the FCAS market to maximize profit. Then, Proximal Policy Optimization, a model-free deep reinforcement learning algorithm, is employed to learn the optimal bidding strategy from the dynamic environment of the energy market under a continuous bidding scale. The proposed model is trained and validated using real-world historical data of the Australian National Electricity Market. The results demonstrate that our developed joint bidding strategy in both markets is significantly profitable compared to individual markets.
翻译:在全球去碳化努力的推动下,可再生能源迅速融入常规电网为参与能源市场的电池能源储存系统带来了新的挑战和机会; 能源套利可以成为BESS收入的一个重要来源,因为可再生能源发电与电力需求不匹配导致现货市场价格波动加剧; 此外,为稳定电网而建立的频率控制辅助服务市场,由于能在毫秒内作出反应的能力,可为BESS提供更高的回报; 因此,BESS必须仔细决定为每个市场分配多少能力,以便在不确定的市场条件下最大限度地增加总利润; 本文将BESS的投标问题作为Markov决策程序,使BESS能够参与现货市场和FCAS市场,以获得最大利润; 然后,为稳定电网而建立的无模型强化深层学习算法,即Proximal政策优化法,用于从持续招标规模的能源市场动态环境中学习最佳的投标战略; 拟议的模型在澳大利亚国家电力市场中,使用实际世界历史数据与我们共同开发的市场相比,展示了澳大利亚国家电力市场。