Posted on Leave a comment

physics assisted GRL for PV voltage regulation


Cao, D., Zhao, J., Hu, J., Pei, Y., Huang, Q., Chen, Z., & Hu, W. (2023). Physics-informed graphical representation-enabled deep reinforcement learning for robust distribution system voltage control. IEEE Transactions on Smart Grid.


This study deals with optimization problems in distribution systems caused by anomalies and flaws in the models and so,It suggests a reliable voltage control method that combines a representation network with a surrogate model DRL. Deep learning approaches extract important characteristics from real-time and pseudo-measurements using graph-based analysis with a tree topology. A soft actor-critic algorithm that was trained on a power flow surrogate model receives these inputs. The technique improves resistance to anomalies and lessens dependency on exact system attributes. Its efficacy is validated on IEEE 33-node and 119-node systems.

Issues in the existing voltage control methods:

  • Because renewable energy is unpredictable, conventional methods have difficulty controlling voltage changes, which might cause grid instability.
  • The inability of stochastic programming (SP) and resilient optimization (RO) techniques to handle load demand and renewable energy output uncertainty limits their usefulness for voltage regulation.
  • Current DRL-based techniques need accurate physical models for training, which presents difficulties in real-world applications because such models could be imprecise or unavailable.
  • The application of many DRL strategies is limited because they presuppose perfect observability of the distribution system, which is sometimes unrealistic in real-world contexts.
  • Existing approaches’ error measures emphasize the need for methods that can function well with partial system observations and lessen need on accurate distribution system models.

proposal of Physical assisted GRL:

Therefore, the author proposes suggests a novel method for controlling the voltage in a resilient distribution system by combining GGAT based surrogate model with DAE and SAC methods.

problem formulation

The author proposes a problem formulation that lays the foundation for tackling strong voltage management in distribution networks using a combination of reinforcement learning and physics-based approaches.

Three different asset categories are used by the system to regulate voltage: energy storage systems (ESS), PV inverters, and static variable compensators (SVCs). For every network node, there are set both active and reactive constrains.

Active and reactive flow constrain formulation is set as:

p_gi_t + p_ei_t – p_ci_t = abs(v_i_t) * sum(abs(v_j_t) * (G_ij * cos(theta_ij) + B_ij * sin(theta_ij)), j=1 to N)

q_gi_t + q_si_t – q_ci_t = abs(v_i_t) * sum(abs(v_j_t) * (G_ij * sin(theta_ij) – B_ij * cos(theta_ij)), j=1 to N)

In this representation, I’ve used more straightforward terms like `abs()` for absolute value, `cos()` and `sin()` for trigonometric functions, and `sum()` to represent summation.

Node voltages are kept within allowable bounds by voltage constraints. To achieve full observability this paper formulates the problem as MDP.

Methodology of physical-assisted GRL:

The methodology for physics assisted multi agent GRL develops a strong voltage control strategy for network distribution by combining surrogate modelling, reinforcement learning methodologies, and neural network topologies in the distribution network, GGAT records structural relationships between nodes. It integrates structural information into the neural network by allocating attention weights to neighboring nodes in accordance with their significance. From the GGAT output the CNN extracts features. DAE reduces the dimensions of the feature extracted from GGAT The Markov Decision Process (MDP), which is used to regulate voltage, is solved by the SAC method using SAC algorithm. A surrogate model uses previous operating data to simulate the power flow calculation process. It contains the voltage prediction module and the GGAT module and The prediction module uses a CNN and fully connected layers to forecast node voltages, while the GGAT module extracts features from pseudo-measurements and real-time data. To training the representation network, the surrogate model’s parameters were optimized under supervision. optimization of these SAC algorithm parameters is based on surrogate model-calculated rewards.

preliminary tests are conducted on IEEE-33 system and the comparative tests are conducted on IEEE-119 some of the major results obtained are as follows:

                          comparison results from the study by Chen, Y(2023)


  • The proposed voltage control technique combines SAC-based control with a physics informed GGAT and DAE-based representation network, providing resilience against anomalous observations.
  • Comparative studies show how effective the approach is against faulty data, preserving control performance throughout a range of anomalous observations.
  • The proposed method reduces dependence on the physical model and achieves performance that is comparable to that determined by precise line parameters.
  • the suggested approach outperforms the conventional stochastic optimization technique in handling rapid voltage fluctuations.
  • Compared to the graph-based policy technique, the suggested method can train a robust voltage control strategy via the supervised learning-aided representation network.


Sakthivel R

I am a First-year M.Sc., AIML student enrolled at SASTRA University

I possess a high level of proficiency in a variety of programming languages and frameworks including Python. I have experience with cloud and database technologies, including SQL, Excel, Pandas, Scikit, TensorFlow, Git, and Power BI

Leave a Reply

Your email address will not be published. Required fields are marked *