Journal of Shanghai Jiao Tong University ›› 2022, Vol. 56 ›› Issue (9): 1262-1275.doi: 10.16183/j.cnki.jsjtu.2021.215
• Mechanical Engineering • Previous Articles Next Articles
LIU Yahui1, SHEN Xingwang1, GU Xinghai1, PENG Tao2, BAO Jinsong1(), ZHANG Dan1
Received:
2021-06-22
Online:
2022-09-28
Published:
2022-10-09
Contact:
BAO Jinsong
E-mail:bao@dhu.edu.cn
CLC Number:
LIU Yahui, SHEN Xingwang, GU Xinghai, PENG Tao, BAO Jinsong, ZHANG Dan. A Dual-System Reinforcement Learning Method for Flexible Job Shop Dynamic Scheduling[J]. Journal of Shanghai Jiao Tong University, 2022, 56(9): 1262-1275.
Add to citation manager EndNote|Ris|BibTeX
URL: https://xuebao.sjtu.edu.cn/EN/10.16183/j.cnki.jsjtu.2021.215
Tab.1
Symbols and variables
符号 | 符号描述 |
---|---|
J | 工件集合 |
G | 设备组集合 |
M | 设备集合 |
P | 人员集合 |
S | 物料集合 |
ok, i, j | 第k个任务中工件Ji的第j道工序 |
j | 工序索引j=1, 2, …, m |
Rk, i, j | 第k个任务中工件Ji的第j道工序配置资源,Rk, i, j={Mk, i, j, Pk, i, j, Sk, i, j} |
Mk, i, j | 工序ok, i, j的配置设备 |
Pk, i, j | 工序ok, i, j的配置操作人员 |
Sk, i, j | 工序ok, i, j的配置物料 |
| 第k个任务中工件Ji的到达时间 |
| 工序ok, i, j的开始时间 |
| 工序ok, i, j的结束时间 |
| 工序ok, i, j与下道工序的准备时间 |
| 第k个任务中工件Ji的总加工时间 |
Wl, t(Wk, i) | 设备组Gl中设备Mt的加工负载(以设备为目标计算得到Wl,t,以工序为单位计算得到Wk,i) |
| 设备组Gl中设备Mt的最大加工负载 |
| 0-1决策变量,取1时表示第k个任务中工件Ji在设备Mi, j上加工 |
DP | 交付期 |
[1] |
JI S X, PAN S R, CAMBRIA E, et al. A survey on knowledge graphs: Representation, acquisition, and applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(2): 494-514.
doi: 10.1109/TNNLS.2021.3070843 URL |
[2] |
ISSA S, ADEKUNLE O, HAMDI F, et al. Knowledge graph completeness: A systematic literature review[J]. IEEE Access, 2021, 9: 31322-31339.
doi: 10.1109/ACCESS.2021.3056622 URL |
[3] |
CARVALHO A, CHOUCHENE A, LIMA T, et al. Cognitive manufacturing in industry 4.0 toward cognitive load reduction: A conceptual framework[J]. Applied System Innovation, 2020, 3(4): 55.
doi: 10.3390/asi3040055 URL |
[4] |
LU Y Q, XU X, WANG L H. Smart manufacturing process and system automation: A critical review of the standards and envisioned scenarios[J]. Journal of Manufacturing Systems, 2020, 56: 312-325.
doi: 10.1016/j.jmsy.2020.06.010 URL |
[5] |
ZHANG J, DING G F, ZOU Y S, et al. Review of job shop scheduling research and its new perspectives under Industry 4.0[J]. Journal of Intelligent Manufacturing, 2019, 30(4): 1809-1830.
doi: 10.1007/s10845-017-1350-2 URL |
[6] | 巴智勇, 袁逸萍, 戴毅, 等. 考虑机器故障的作业车间调度方案鲁棒测度方法[J]. 计算机集成制造系统, 2020, 26(12): 3341-3349. |
BA Zhiyong, YUAN Yiping, DAI Yi, et al. Robustness measurement approach of job shop scheduling with machine breakdowns[J]. Computer Integrated Manufacturing Systems, 2020, 26(12): 3341-3349. | |
[7] | 方佳, 陆志强. 考虑设备故障的鲁棒调度计划模板的建模优化[J]. 上海交通大学学报, 2020, 54(12): 1278-1290. |
FANG Jia, LU Zhiqiang. Modeling and optimization of robust scheduling template considering equipment failure[J]. Journal of Shanghai Jiao Tong University, 2020, 54(12): 1278-1290. | |
[8] |
GAO K Z, YANG F J, ZHOU M C, et al. Flexible job-shop rescheduling for new job insertion by using discrete jaya algorithm[J]. IEEE Transactions on Cybernetics, 2019, 49(5): 1944-1955.
doi: 10.1109/TCYB.2018.2817240 pmid: 29993706 |
[9] | 王鹏飞. 群智能优化算法及在流水车间调度问题中的应用研究[D]. 长春: 吉林大学, 2019. |
WANG Pengfei. Swarm intelligence optimization algorithm and its application in flow shop scheduling problem[D]. Changchun: Jilin University, 2019. | |
[10] | ZHOU B, BAO J S, LI J, et al. A novel knowledge graph-based optimization approach for resource allocation in discrete manufacturing workshops[J]. Robotics and Computer-Integrated Manufacturing, 2021, 71: 102160. |
[11] | CHAKRABORTTY R K, RAHMAN H F, RYAN M J. Efficient priority rules for project scheduling under dynamic environments: A heuristic approach[J]. Computers & Industrial Engineering, 2020, 140: 106287. |
[12] | 蒋小康, 张朋, 吕佑龙, 等. 基于混合蚁群算法的半导体生产线炉管区调度方法[J]. 上海交通大学学报, 2020, 54(8): 792-804. |
JIANG Xiaokang, ZHANG Peng, LYU Youlong, et al. Hybrid ant colony algorithm for batch scheduling in semiconductor furnace operation[J]. Journal of Shanghai Jiao Tong University, 2020, 54(8): 792-804. | |
[13] | 王金凤, 陈璐, 杨雯慧. 考虑设备可用性约束的单机调度问题[J]. 上海交通大学学报, 2021, 55(1): 103-110. |
WANG Jinfeng, CHEN Lu, YANG Wenhui. A single machine scheduling problem considering machine availability constraints[J]. Journal of Shanghai Jiao Tong University, 2021, 55(1): 103-110. | |
[14] | 杜轩, 潘志成. 聚类差分进化算法求解多目标工艺规划与调度集成问题[J]. 计算机集成制造系统, 2019, 25(7): 1729-1738. |
DU Xuan, PAN Zhicheng. Clustering and differential evolution algorithm for solving multi-objectives IPPS problem[J]. Computer Integrated Manufacturing Systems, 2019, 25(7): 1729-1738. | |
[15] |
李聪波, 沈欢, 李玲玲, 等. 面向能耗的多工艺路线柔性作业车间分批优化调度模型[J]. 机械工程学报, 2017, 53(5): 12-23.
doi: 10.3901/JME.2017.05.012 |
LI Congbo, SHEN Huan, LI Lingling, et al. A batch splitting flexible job shop scheduling model for energy saving under alternative process plans[J]. Journal of Mechanical Engineering, 2017, 53(5): 12-23.
doi: 10.3901/JME.2017.05.012 |
|
[16] | PENG C, WU G L, LIAO T W, et al. Research on multi-agent genetic algorithm based on tabu search for the job shop scheduling problem[J]. PLoS One, 2019, 14(9): e0223182. |
[17] |
KUNDAKCI N, KULAK O. Hybrid genetic algorithms for minimizing makespan in dynamic job shop scheduling problem[J]. Computers & Industrial Engineering, 2016, 96: 31-51.
doi: 10.1016/j.cie.2016.03.011 URL |
[18] |
SHEN X N, YAO X. Mathematical modeling and multi-objective evolutionary algorithms applied to dynamic flexible job shop scheduling problems[J]. Information Sciences, 2015, 298: 198-224.
doi: 10.1016/j.ins.2014.11.036 URL |
[19] | WANG Z, ZHANG J H, YANG S X. An improved particle swarm optimization algorithm for dynamic job shop scheduling problems with random job arrivals[J]. Swarm and Evolutionary Computation, 2019, 51: 100594. |
[20] | 张洁, 张朋, 刘国宝. 基于两阶段蚁群算法的带非等效并行机的作业车间调度[J]. 机械工程学报, 2013, 49(6): 136-144. |
ZHANG Jie, ZHANG Peng, LIU Guobao. Two-stage ant colony algorithm based job shop scheduling with unrelated parallel machines[J]. Journal of Mechanical Engineering, 2013, 49(6): 136-144. | |
[21] | 周亚勤, 杨长祺, 吕佑龙, 等. 双资源约束的航天结构件车间生产调度方法[J]. 机械工程学报, 2018, 54(9): 55-63. |
ZHOU Yaqin, YANG Changqi, LÜ Youlong, et al. Scheduling the production of aerospace structural parts with dual resource constraints[J]. Journal of Mechanical Engineering, 2018, 54(9): 55-63.
doi: 10.3901/JME.2018.09.055 |
|
[22] | 汪浩祥, 严洪森, 汪峥. 知识化制造环境中基于双层Q学习的航空发动机自适应装配调度[J]. 计算机集成制造系统, 2014, 20(12): 3000-3010. |
WANG Haoxiang, YAN Hongsen, WANG Zheng. Adaptive assembly scheduling of aero-engine based on double-layer Q-learning in knowledgeable manufacturing[J]. Computer Integrated Manufacturing Systems, 2014, 20(12): 3000-3010. | |
[23] |
WEI Y, PAN L, LIU S J, et al. DRL-scheduling: An intelligent QoS-aware job scheduling framework for applications in clouds[J]. IEEE Access, 2018, 6: 55112-55125.
doi: 10.1109/ACCESS.2018.2872674 URL |
[24] |
WANG Y D, LIU H, ZHENG W B, et al. Multi-objective workflow scheduling with deep-Q-network-based multi-agent reinforcement learning[J]. IEEE Access, 2019, 7: 39974-39982.
doi: 10.1109/ACCESS.2019.2902846 URL |
[25] | LUO S. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning[J]. Applied Soft Computing, 2020, 91: 106208. |
[26] |
HE Z L, TRAN K P, THOMASSEY S, et al. Multi-objective optimization of the textile manufacturing process using deep-Q-network based multi-agent reinforcement learning[J]. Journal of Manufacturing Systems, 2022, 62: 939-949.
doi: 10.1016/j.jmsy.2021.03.017 URL |
[27] | 林时敬, 徐安军, 刘成, 等. 基于深度强化学习的炼钢车间天车调度方法[J]. 中国冶金, 2021, 31(3): 37-43. |
LIN Shijing, XU Anjun, LIU Cheng, et al. Crane scheduling method in steelmaking workshop based on deep reinforcement learning[J]. China Metallurgy, 2021, 31(3): 37-43. | |
[28] |
BRANDIMARTE P. Routing and scheduling in a flexible job shop by tabu search[J]. Annals of Operations Research, 1993, 41(3): 157-183.
doi: 10.1007/BF02023073 URL |
[29] | 喻鹏, 张俊也, 李文璟, 等. 移动边缘网络中基于双深度Q学习的高能效资源分配方法[J]. 通信学报, 2020, 41(12): 148-161. |
YU Peng, ZHANG Junye, LI Wenjing, et al. Energy-efficient resource allocation method in mobile edge network based on double deep Q-learning[J]. Journal on Communications, 2020, 41(12): 148-161. | |
[30] | 牟乃夏, 徐玉静, 李洁, 等. 遗传禁忌搜索算法收敛性和时间复杂度分析[J]. 河南理工大学学报(自然科学版), 2018, 37(4): 118-122. |
MOU Naixia, XU Yujing, LI Jie, et al. Analyses of convergence and time complexity of genetic tabu search algorithm[J]. Journal of Henan Polytechnic University (Natural Science), 2018, 37(4): 118-122. |
[1] | JI Xiukun (冀秀坤), HAI Jintao (海金涛), LUO Wenguang (罗文广), LIN Cuixia (林翠霞), XIONG Yu(熊 禹), OU Zengkai (殴增开), WEN Jiayan(文家燕). Obstacle Avoidance in Multi-Agent Formation Process Based on Deep Reinforcement Learning [J]. J Shanghai Jiaotong Univ Sci, 2021, 26(5): 680-685. |
[2] | SONG Dengqiang, ZHOU Bin, SHEN Xingwang, BAO Jinsong, ZHOU Yaqin. Dynamic Knowledge Graph Modeling Method for Ship Block Manufacturing Process [J]. Journal of Shanghai Jiao Tong University, 2021, 55(5): 544-556. |
[3] | LI Biao, WANG Liwen, XING Zhiwei, WANG Sibo, LUO Qian. Cooperative Control of Aircraft Ground Deicing Resources [J]. Journal of Shanghai Jiao Tong University, 2021, 55(11): 1362-1370. |
[4] | CHEN Zhiyu, BAO Jinsong, ZHENG Xiaohu, LIU Tianyuan . Assembly Information Model Based on Knowledge Graph [J]. J Shanghai Jiaotong Univ Sci, 2020, 25(5): 578-588. |
[5] | TANG Xianlun (唐贤伦), LIU Nianci (刘念慈), WAN Yali (万亚利), GUO Fei (郭飞). Multi-Step Model Predictive Control Based on Online Support Vector Regression Optimized by Multi-Agent Particle Swarm Optimization Algorithm [J]. Journal of Shanghai Jiao Tong University (Science), 2018, 23(5): 607-612. |
[6] | ZHANG Ye,JIA Yuting,FU Luoyi,WANG Xinbing. AceMap Academic Map and AceKG Academic Knowledge Graph for Academic Data Visualization [J]. Journal of Shanghai Jiaotong University, 2018, 52(10): 1357-1362. |
[7] | QIU Lu1,YE Yinzhong2. A Multi-Agent System Based Fault Diagnostic Approach for Topological Structure of Dynamic Networking System [J]. Journal of Shanghai Jiaotong University, 2015, 49(06): 902-906. |
[8] | ZHANG Xu,CHEN Maoyin,WANG Ling,ZHOU Donghua. Fault-Tolerant Consensus for a Network of Multi-Agent Systems with Actuator Faults [J]. Journal of Shanghai Jiaotong University, 2015, 49(06): 806-811. |
[9] | QIN Jiu-Feng, ZENG Fan-Ming, CHEN Yu-Tao. Method of Multi-agent Autonomous State Reconfiguration of Shipboard Chilled Water System [J]. Journal of Shanghai Jiaotong University, 2013, 47(02): 284-289. |
[10] | LIN Haiping1,2,TAN Xiaohong1,SHEN Ruimin1. A Building Algorithm for Individual Learning Content Basedon Structural Knowledge Graph [J]. Journal of Shanghai Jiaotong University, 2010, 44(03): 418-0422. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||