Posted on Leave a comment

Graph-Based RL for Dynamic Network Scheduling


Xing, Q., Chen, Z., Zhang, T., Li, X., & Sun, K. (2023). Real-time optimal scheduling for active distribution networks: A graph reinforcement learning method. International Journal of Electrical Power & Energy Systems145, 108637.


By connecting ADN with GAT (Graphical Attention Network) and DDPG (Deep Deterministic Policy Gradient), the author worked on providing a real-time online collaborative optimization of control equipment for improved economy and safety of ADNs (active distribution networks). The GAT extracts complex unstructured graphical information that contains power information in the form of nodes and topological relations in the form of edges that come from ADN and feeds these feature extractions back into ADN as action output by passing them into a DRL (Deep Reinforcement Learning) and DPDG generates optimization solutions for efficient scheduling of ADN. Case studies are conducted by contrasting the suggested GRL model with conventional DRL approaches. 

 Limitations in other papers in scheduling network:

Uncertain random output generations are produced by the high connectedness of distributive generations found in modern distributive generations like ADN.  Certain papers offer research on the best ADN policies; however, they rely on precise prediction data, such outputs from renewable energy sources, and neglect to account for source-load uncertainty in their modelling study. The precision of the prediction data has a significant impact on the optimization outcomes. Making quick judgments on online dispatching for the growing amount of controllable equipment is challenging where Recent development of DRL provides a solution for computer complexity of ADN’s and the strong connection of distributive generations seen in contemporary distributive generations such as ADN results in uncertain random output generations.
Some studies provide research on optimal ADN policies, but they base their analysis on accurate forecast data, such outputs from renewable energy sources, and thus don’t take source-load uncertainty into consideration when modelling. The optimization results are significantly influenced by the prediction data’s accuracy. It is difficult to make snap decisions on online dispatching for the increasing quantity of controllable equipment. 
none of the previous work sufficiently evaluates the feasibility and scalability of DRL-based optimal dispatch models for ADNs under the topology variation scenarios. 


nevertheless, there are still two primary problems that come up, like:
• To begin with, the majority of DRL-based ADN optimization techniques (such as DQN and DDPG) fall short of fully examining the organic graph-structured characteristics present in distribution systems.
• Changes in topology and operating patterns brought on by emergencies frequently happen in the distribution 



 structure of the proposed scheduling for ADNs from the study by Xing, Q., Chen, Z., Zhang, T., Li, X., & Sun, K. (2023)

The author’s technique entails that taking unstructured graphical data out of ADN and putting it into a DDPG, which then generates an action output for the best possible scheduling. The ADNs use this action output to make the best decisions possible, increasing the ADN’s economy and safety. 

Eess t,k = Eess t− 1,k + v ess,ch t,k ηess,ch k Δt Qess k Pess,ch t,k + v ess,ch t,k Δt ηess,disc k Qess k Pess,disc t,k Eess,min  

k EESS t,k Eess,max k 

 Eess,end k = Eess,pref 

, this equation assures that the state of charge (SOC) of an ESS is within the operating range so, avoiding.
 the damage to batteries resulting from excessive charging and discharging for the best possible model scheduling. the harm that comes from overcharging and overdischarging batteries in order to achieve optimal model scheduling. 

The author then formulates the energy management decision-making issue as an FMDP using the features that were taken from GAT. The primary variables and elements of decision-making included in the FMDP include states, actions, and rewards. 

conclusion :

  • The technique for optimizing ADNs that combines GAT and DDPG is presented in the paper: GRL.
  • GRL aims for distribution networks to operate safely and economically.
  •  GAT extracts ADNs’ load characteristics and topological structure.
  • For controlled equipment, DDPG develops energy management plans.
  • Under steady conditions, the results demonstrate lower operating costs and better decision-making.
  • In cases of unknown faults, GRL performs better than DDPG and GDDPG.
  • One area of future research will be GNN embedding in reinforcement learning.


Sakthivel R

I am a First-year M.Sc., AIML student enrolled at SASTRA University

I possess a high level of proficiency in a variety of programming languages and frameworks including Python. I have experience with cloud and database technologies, including SQL, Excel, Pandas, Scikit, TensorFlow, Git, and Power BI

Leave a Reply

Your email address will not be published. Required fields are marked *