Journal of Shanghai Jiao Tong University ›› 2022, Vol. 56 ›› Issue (9): 1262-1275.doi: 10.16183/j.cnki.jsjtu.2021.215

• Mechanical Engineering • Previous Articles     Next Articles

A Dual-System Reinforcement Learning Method for Flexible Job Shop Dynamic Scheduling

LIU Yahui1, SHEN Xingwang1, GU Xinghai1, PENG Tao2, BAO Jinsong1(), ZHANG Dan1   

  1. 1. School of Mechanical Engineering, Donghua University, Shanghai 201620, China
    2. School of Mechanical Engineering, Zhejiang University, Hangzhou 310027, China
  • Received:2021-06-22 Online:2022-09-28 Published:2022-10-09
  • Contact: BAO Jinsong E-mail:bao@dhu.edu.cn

Abstract:

In the production process of aerospace structural parts, there coexist batch production tasks and research and development (R&D) tasks. Personalized small-batch R&D and production tasks lead to frequent emergency insertion orders. In order to ensure that the task is completed on schedule and to solve the flexible job shop dynamic scheduling problem, this paper takes minimization of equipment average load and total completion time as optimization goals, and proposes a dual-loop deep Q network (DL-DQN) method driven by a perception-cognition dual system. Based on the knowledge graph, the perception system realizes the representation of workshop knowledge and the generation of multi-dimensional information matrix. The cognitive system abstracts the scheduling process into two stages: resource allocation agent and process sequencing agent, corresponding to two optimization goals respectively. The workshop status matrix is designed to describe the problems and constraints. In scheduling decision, action instructions are introduced step by step. Finally, the reward function is designed to realize the evaluation of resource allocation decision and process sequence decision. Application of the proposed method in the aerospace shell processing of an aerospace institute and comparative analysis of different algorithms verify the superiority of the proposed method.

Key words: perception-cognition dual system, dual-loop deep Q network (DL-DQN), dynamic scheduling, knowledge graph, multi-agent

CLC Number: