J Shanghai Jiaotong Univ Sci ›› 2021, Vol. 26 ›› Issue (5): 680-685.doi: 10.1007/s12204-021-2357-6

• Intelligent Connected Vehicle • Previous Articles     Next Articles

Obstacle Avoidance in Multi-Agent Formation Process Based on Deep Reinforcement Learning

JI Xiukun1 (冀秀坤), HAI Jintao1 (海金涛), LUO Wenguang1 (罗文广), LIN Cuixia1 (林翠霞), XIONG Yu2 (熊 禹), OU Zengkai2 (殴增开), WEN Jiayan1 (文家燕)   

  1. (1. Guangxi Key Laboratory of Auto Parts and Vehicle Technology; School of Electrical and Information Engineering, Guangxi University of Science and Technology, Liuzhou 545006, Guangxi, China; 2. Technology Center of Dongfeng Liuzhou Automobile Co., Ltd., Liuzhou 545000, Guangxi, China)
  • Received:2020-11-28 Online:2021-10-26 Published:2021-10-28

Abstract: To solve the problems of di?cult control law design, poor portability, and poor stability of traditional multi-agent formation obstacle avoidance algorithms, a multi-agent formation obstacle avoidance method based on deep reinforcement learning (DRL) is proposed. This method combines the perception ability of convolutional neural networks (CNNs) with the decision-making ability of reinforcement learning in a general form and realizes direct output control from the visual perception input of the environment to the action through an end-to-end learning method. The multi-agent system (MAS) model of the follow-leader formation method was designed with the wheelbarrow as the control object. An improved deep Q netwrok (DQN) algorithm (we improved its discount factor and learning e?ciency and designed a reward value function that considers the distance relationship between the agent and the obstacle and the coordination factor between the multi-agents) was designed to achieve obstacle avoidance and collision avoidance in the process of multi-agent formation into the desired formation. The simulation results show that the proposed method achieves the expected goal of multi-agent formation obstacle avoidance and has stronger portability compared with the traditional algorithm.

Key words: wheelbarrow, multi-agent, deep reinforcement learning (DRL), formation, obstacle avoidance

CLC Number: