To solve the problems of di?cult control law design, poor portability, and poor stability of traditional multi-agent formation obstacle avoidance algorithms, a multi-agent formation obstacle avoidance method based on deep reinforcement learning (DRL) is proposed. This method combines the perception ability of convolutional neural networks (CNNs) with the decision-making ability of reinforcement learning in a general form and realizes direct output control from the visual perception input of the environment to the action through an end-to-end learning method. The multi-agent system (MAS) model of the follow-leader formation method was designed with the wheelbarrow as the control object. An improved deep Q netwrok (DQN) algorithm (we improved its discount factor and learning e?ciency and designed a reward value function that considers the distance relationship between the agent and the obstacle and the coordination factor between the multi-agents) was designed to achieve obstacle avoidance and collision avoidance in the process of multi-agent formation into the desired formation. The simulation results show that the proposed method achieves the expected goal of multi-agent formation obstacle avoidance and has stronger portability compared with the traditional algorithm.
JI Xiukun (冀秀坤), HAI Jintao (海金涛), LUO Wenguang (罗文广), LIN Cuixia (林翠霞), XIONG Yu(熊 禹), OU Zengkai (殴增开), WEN Jiayan(文家燕)
. Obstacle Avoidance in Multi-Agent Formation Process Based on Deep Reinforcement Learning[J]. Journal of Shanghai Jiaotong University(Science), 2021
, 26(5)
: 680
-685
.
DOI: 10.1007/s12204-021-2357-6
[1] XIE G, ZHANG Y. Survey of consensus problem in cooperative control of multi-agent systems [J]. Appli-cation Research of Computers, 2011, 28(6): 2035-2039 (in Chinese).
[2] CHEN Z, LIN L, YAN G. An approach to scienti?c cooperative robotics: Through MAS (multi-agent sys-tem) [J]. Robot, 2001, 23(4): 368-373 (in Chinese). [3] DUAN Y, YANG H, CUI B, et al. Application of re-inforcement learning to basic action learning of soccer robot [J]. Robot, 2008, 30(5): 453-459 (in Chinese).
[4] LITTMAN M L. Reinforcement learning improves be-haviour from evaluative feedback [J]. Nature, 2015, 521(7553): 445-451.
[5] ZHU Y, ZHAO D. Probably approximately correct re-inforcement learning solving continuous-state control problem [J]. Control Theory and Applications, 2016, 33(12): 1603-1613 (in Chinese).
[6] ZHOU W. The application of deep learning algo-rithms in intelligent collaborative robots [J]. China New Telecommunications, 2017, 19(21): 129-130 (in Chinese).
[7] POLYDOROS A S, NALPANTIDIS L. Survey of model-based reinforcement learning: Applications on robotics [J]. Journal of Intelligent & Robotic Systems, 2017, 86(2): 153-173.
[8] LIMA H, KUROE Y. Swarm reinforcement learning methods improving certaintyof learningfor amulti-robot formation problem [C]//2015 IEEE Congress on Evolutionary Computation (CEC). Sendai: IEEE, 2015: 3026-3033.
[9] LIU Q, ZHAI J, ZHANG Z, et al. A survey on deep reinforcement learning [J]. Chinese Journal of Com-puters, 2018, 41(1): 1-27 (in Chinese).
[10] RIEDMILLER M. Neural ?tted Q iteration: First ex-periences with a data e?cient neural reinforcement learning method [M]//Machine learning: ECML2005. Berlin, Heidelberg: Springer, 2005: 317-328.
[11] LANGE S, RIEDMILLER M. Deep auto-encoder neu-ral networks in reinforcement learning [C]//The 2010 International Joint Conference on Neural Networks (IJCNN). Barcelona: IEEE, 2010: 1-8.
[12] ABTAHI F, FASEL I. Deep belief nets as func-tion approximators for reinforcement learning [C]//Workshops at the Twenty-Fifth AAAI Confer-ence on Arti?cial Intelligence. Frankfurt: AAAI, 2011:
2- 7.
