Posted on Leave a comment

Job scheduling in manufacturing systems using GRL.


Liu, Z., Mao, H., Sa, G., Liu, H., & Tan, J. (2024). Dynamic job-shop scheduling using graph reinforcement learning with auxiliary strategy. Journal of Manufacturing Systems73, 1-18.


Unpredictable dynamic events in manufacturing systems forms as a limitation to effective job-shop scheduling (JSP) additionally The efficiency of solutions and dynamic adaptation are difficult to balance in present methods. To tackle this the author of this paper suggests an approach for dynamic job-shop scheduling (DJSP) using GRL. The state representation is expanded upon, a mixed graph Transformer network for adapting changes, and a novel Phase Proximal Policy Optimization algorithm with Rollback is developed in this paper

Limitations in the existing DJSP methods:

Although the goal of robust scheduling in the earlier works was to maintain performance in increasing uncertainty and it often involves resource redundancy and could not consider all perturbations effectively reactive scheduling resulted in increased computational cost.

PDRs method suggested by other papers put low computational effort, but it is not ideal and can be strongly impacted by instance-related factors. Previous attempts which use DRL for DSJP problems had lack of generalization across various instance. Although to tackle this some other papers utilize graph-based DRL methods provide valuable insights, they might not be able to cope with the growing diversity of instances and complexity of environments.


To overcome these issues the author proposes a GRL-AS framework.

Therefore, the author proposes a problem formulation that provides the complexity of DJSP through considering uncertainties in processing time and machine failure with the objective to improve efficiency of manufacturing.

The reward function goal is determined by taking the average machine usage and which is defined as:


The methodology combines graph representation and reinforcement learning, also auxiliary strategies to effectively address the challenges of dynamic job-shop scheduling with stochastic processing times and machine breakdowns.


General complete GRL-AS framework for solving DJSPs illustrated from by J. Chen (2021)

The scheduling problem is represented by disjunctive graphs. Operations are represented by nodes, priority restrictions are represented by edges, and operations that are carried out on the same machine are represented by disjunctions. The objective is to minimize the length of the longest path in the directed acyclic graph. The MGTN be used to transform disjunctive networks into fixed-length feature vectors in order to efficiently capture the topological and feature interactions between nodes in the graph. The graph representation module and reinforcement learning technique are combined in the Graph Reinforcement Learning with Auxiliary Strategy (GRL-AS) framework.

Presently There are two parts to it: online deployment and offline learning. In order to train the RL agent offline, DJSP instances are transformed into separate graphs and transmitted into the graph representation module additionally the Gradient algorithms are used by the agent to constantly learn the best scheduling strategies and an P3OR algorithm is used to improve the stability once the model is trained it can be provided for online applications.


  • Therefore, proposed a GRL-AS framework to manifesting system’s DSJP issues.
  • MGTN is used for transforming separate disjoined network into fixed length.
  • P3OR algorithm utilized for stable training.
  • The proposed model shows a better performance when handling both static as well as dynamic DJSP instances.


Sakthivel R

I am a First-year M.Sc., AIML student enrolled at SASTRA University

I possess a high level of proficiency in a variety of programming languages and frameworks including Python. I have experience with cloud and database technologies, including SQL, Excel, Pandas, Scikit, TensorFlow, Git, and Power BI

Leave a Reply

Your email address will not be published. Required fields are marked *