Posted on Leave a comment

Graph RL for High renewable Energy penetration


Li, P., Huang, W., Dai, Z., Hou, J., Cao, S., Zhang, J., & Chen, J. (2022, March). A Novel Graph Reinforcement Learning Approach for Stochastic Dynamic Economic Dispatch under High Penetration of Renewable Energy. In 2022 4th Asia Energy and Electrical Engineering Symposium (AEEES) (pp. 498-503). IEEE.


In order to enhance the decision quality of economic dispatch, the author of this work suggests using graph reinforcement learning (GRL) to sort out the improved uncertainty produced by a large number of distributive generations and next the GRL more accurately depicts the structure of the system. Because of the constraints in his previous work the author inspired to produce this one, which employs GRL to get around those limits.

The limitations from other studies: 

  • MIQP method, which significantly raises the standard of ED’s global optimum solution search. Moreover, some researchers attempt to combine heuristic algorithms with MILP and put out novel concepts as EP-SQP, PSO-SQP, AIS-SQP, and so on.  
  •  Heuristic algorithms are severely constrained by their poor optimization efficiency and sensitivity to parameters, whereas MILP depends on accurate system modeling, which is challenging to put into practice. 
  •  Heuristic algorithms are highly limited by their low optimization efficiency and parameter sensitivity, while MILP relies on precise system modeling, which is difficult to implement. 


So, for these limitations the author proposes GRL, which is a graph convolutional network (GCN) with extension of soft actor-critic (SAC). Because GRL is built on SAC, and it has SAC’s capacity to prevent local optimum while making decisions. By using entropy regularization, learning progresses more quickly.
GRL only requires real-time operational circumstances when the algorithm is executing online in order to achieve extremely quick decision making. The updated IEEE39 instance validates the validity and speed of GRL.

Methodology of validating GRL using modified IEEE39:  

Node characteristics are specified as node operation features in this study, and they may be represented as Pload, Qload, PG, P WT, P PV, E BS, t, a]. s. Topological structure is used as the graph structure. f is equivalent to 8 in this paper and Using the IEEE5 scenario as an illustration, the following figure: 

                                figure of Power system operation state graph representation from the study by Li et al. (2022)

In order to extract the structural characteristics of graph data, this article first adopts a fully connected layer for feature transformation and then employs a two-layer graph convolutional neural network for stacking. The nonlinear mapping of the system state to action policy is then realized by using three complete connection layers which is shown below in the picture:


 connection layer of the proposed framework from the study by Li et al. (2022)

IEEE39 framework and working:

As seen in the figure below, a modified IEEE39 is used for a 24-hour real-time operation with 10 conventional generations, 2 WTs, 2 PVs, and 1 BS. There are 96 phases in a day of the real-time dispatching cycle, which is scheduled to run every 15 minutes. It is presumed that all generations—conventional and unconventional—are always in usable shape. The designated training hours are 8000 where as Python is used for all simulations in this, and it has an Inter Core i7 processor and 16GB of RAM. 


                       IEEE39 structure from the study by Li et al. (2022)

8,000 days of training trials, with 96 stages each day and 768,000 data points, were carried out for this work and In this instance and the generating parameters are displayed in the TABLE below there Both βBS and βBS are fixed at $100/Mw*h; the cost of the power system purchasing electricity from the external grid is $300/Mw*h, while the cost of selling the power is just $30/Mw*h.

Evaluation results from the study by Li et al. (2022)


  • A novel method for GRL was created with a high penetration rate of RES for dynamic economic dispatch.
  • GRL uses system topology to aggregate operating data.
  • The depiction of operation data using graphs reveals strong relationships between them.
  • GRL increases system flexibility and economy.
  • shorter offline pre-learning period than with current techniques.
  • Because the state graph is sparse, the algorithm has strong scalability.
  • Future scope: promising potential for use in high-renewable energy systems. 


  •  GRL: Graph Reinforcement Learning
  • RES: Renewable Energy Sources
  • GCN: Graph Convolutional Network
  • SAC: Soft Actor-Critic
  • WT: Wind Turbine
  • PV: Photovoltaic
  • BS: Base Station
  • RE: Renewable Energy

Sakthivel R

I am a First-year M.Sc., AIML student enrolled at SASTRA University

I possess a high level of proficiency in a variety of programming languages and frameworks including Python. I have experience with cloud and database technologies, including SQL, Excel, Pandas, Scikit, TensorFlow, Git, and Power BI

Leave a Reply

Your email address will not be published. Required fields are marked *