The prevalence of the Internet of things (IoT) and smart meters devices in smart grids is providing key support for measuring and analyzing the power consumption patterns. This approach enables end-user to play the role of prosumers in the market and subsequently contributes to diminish the carbon footprint and the burden on utility grids. The coordination of trading surpluses of energy that is generated by house renewable energy resources (RERs) and the supply of shortages by external networks (main grid) is a necessity. This paper proposes a hierarchical architecture to manage energy in multiple smart buildings leveraging federated deep reinforcement learning (FDRL) with dynamic load in a distributed manner. Within the context of the developed FDRL-based framework, each agent that is hosted in local building energy management systems (BEMS) trains a local deep reinforcement learning (DRL) model and shares its experience in the form of model hyperparameters to the federation layer in the energy management system (EMS). Simulation studies are conducted using one EMS and up to twenty smart houses that are equipped with photovoltaic (PV) systems and batteries. This iterative training approach enables the proposed discretized soft actor-critic (SAC) agents to aggregate the collected knowledge to expedite the overall learning procedure and reduce costs and CO2 emissions, while the federation approach can mitigate privacy breaches. The numerical results confirm the performance of the proposed framework under different daytime periods, loads, and temperatures.
翻译:智能电网中物质(IoT)和智能仪表装置的普及性为测量和分析电力消费模式提供了关键支持。这一方法使最终用户能够在市场中扮演期票人的角色,进而帮助减少碳足迹和电网负担。协调由室内可再生能源资源(RERs)产生的能源贸易剩余量和外部网络(主电网)供应的能源短缺是一个必要因素。本文件建议建立一个等级结构,在多家智能建筑中管理能源,利用联合的深度加热学习(FDRL),以分散方式进行动态负荷。在基于FDRL的发达框架内,由当地建筑能源管理系统(BEMS)托管的每个代理机构都培训当地深度加固学习(DRL)模式,并分享其在能源管理系统(EMS)中以模型超光电计的形式向联邦系统(主网)供应的能源。模拟研究利用一个EMS和多达20所装有光电(PV)系统和电池的智能房进行能源管理。这种迭代培训方法使拟议的离式软操作员-credial-credial commainal acilal acilal cust the coil the coil the coll coll the coll the coll the coil the cust the untical dal dal dal commocuildal compal compal compal uncal unal unal uncal uncal uncal uncal uncal uncal uncal uncal uncal uncal uncal uncal uncal 和Ctors uncil uncil 和Cal 和Cal uncres可以降低整个化的快速化的快速化程序,可以降低成本,可以降低成本和Ctoral 和Ctoral 和Creal acal acal 和Creal 和Creal acal 方法,可以降低 和Crestical 和Creal acal 方法,可以降低成本,并可以降低成本,并可以降低成本,并可以使数制制制可降低成本,并可以降低成本,并可以确认制制制制