Abstract:

Aiming at the scheduling problem of welding flow shops considering equipment preventive maintenance, with the optimization objective of minimizing the maximum completion time, and addressing such difficulties as equipment failure shocks, limited buffers, and ambiguous representation caused by high-dimensional state spaces, this paper proposes a joint optimization method for scheduling and maintenance based on representation-enhanced multi-agent reinforcement learning. The problem is decomposed into two sub-problems: processing scheduling and preventive maintenance, and a scheduling-maintenance dual-agent architecture is constructed. Given the strong coupling between these sub-problems in which scheduling influences equipment failure risks and maintenance alters equipment availability, the dual agents leverage the value-decomposition multi-agent actor-critics (VDAC) algorithm to decompose the global value function into their respective local value functions. This allows both agents to naturally incorporate considerations of each other’s sub-problems when optimizing their own local objectives, thereby enabling collaborative problem-solving. Representation enhancement extracts key information from high-dimensional states through an autoencoder, resolving the problems of information redundancy and ambiguous representation in high-dimensional state spaces, allowing the agents to make decisions based on key representational information and improving the performance of joint scheduling and maintenance optimization. Case studies show that compared with other algorithms, the minimized maximum completion time is reduced by an average of 4.13%, and by an average of 13.34% compared with rule-based algorithms.

Key words:  , flow shop, multi-agent reinforcement learning, limited buffer, preventive maintenance, representation enhancement