Missile-Target Situation Assessment Model Based on Reinforcement Learning

doi:10.1007/s12204-020-2226-8

Abstract

Abstract: In situation assessment (SA) of missile versus target fighter, the traditional SA models generally
have the characteristics of strong subjectivity and poor dynamic adaptability. This paper considers SA as an
expectation of future returns and establishes a missile-target simulation battle model. The actor-critic (AC)
algorithm in reinforcement learning (RL) is used to train the evaluation network, and a missile-target SA model
is established in simulation battle training. Simulation and comparative experiments show that the model can
effectively estimate the expected effect of missile attack under the current situation, and it provides an effective
basis for missile attack decision.

Key words: situation assessment (SA)| battle model| reinforcement learning (RL)| actor-critic (AC) algorithm

摘要： In situation assessment (SA) of missile versus target fighter, the traditional SA models generally
have the characteristics of strong subjectivity and poor dynamic adaptability. This paper considers SA as an
expectation of future returns and establishes a missile-target simulation battle model. The actor-critic (AC)
algorithm in reinforcement learning (RL) is used to train the evaluation network, and a missile-target SA model
is established in simulation battle training. Simulation and comparative experiments show that the model can
effectively estimate the expected effect of missile attack under the current situation, and it provides an effective
basis for missile attack decision.

关键词: situation assessment (SA)| battle model| reinforcement learning (RL)| actor-critic (AC) algorithm

CLC Number:

TP 391
TP 18

ZHANG Yun, Lü Runyan, CAI Yunze . Missile-Target Situation Assessment Model Based on Reinforcement Learning[J]. Journal of Shanghai Jiao Tong University(Science), 2020, 25(5): 561-568.

References 16

[1]	ENDSLEY M R. Toward a theory of situation awareness in dynamic systems [J]. Human Factors, 1995,37(1): 32-64.
[2]	STEINBERG A N, BOWMAN C L,WHITE F E. Revisions to the JDL data fusion model [J]. Proceedings of SPIE, 1999, 3719: 430-441.
[3]	CHEN X, WEI X M, XU G Y. Multiple unmanned aerial vehicle decentralized cooperative air combat decision making with fuzzy situation [J]. Journal of Shanghai Jiao Tong University, 2014, 48(7): 907-913(in Chinese).
[4]	YAN C C, HAO Y S. Threat assessment of aerial target based on AHP [J]. Computing Technology and Automation,2011, 30(2): 118-121 (in Chinese).
[5]	CHEN J, YU G H, GAO X G. Cooperative threat assessment of multi-aircrafts based on synthetic fuzzy cognitive map [J]. Journal of Shanghai Jiao Tong University(Science), 2012, 17(2): 228-232.
[6]	LU C G, ZHOU Z L, LIU H Q, et al. Situation assessment of far-distance attack air combat based on mixed dynamic bayesian networks [C]//Proceedings of the 37th Chinese Control Conference. Wuhan, China:Chinese Association of Automation, 2018: 1133-1138.
[7]	PENG P, WEN Y, YANG Y D, et al. Multiagent bidirectionally-coordinated nets: Emergence of human-level coordination in learning to play Star-Craft combat games [EB/OL]. (2017-09-14) [2020-07-14]. https://arxiv.org/pdf/1703.10069v4.pdf.
[8]	ZHOU Z Q, QIAN J G, WANG Y Z. Research on ballistic missile situation grade model based on BP neural network [J]. Fire Control & Command Control, 2015,40(5): 53-56 (in Chinese).
[9]	LIU P, MA Y F. A deep reinforcement learning based intelligent decision method for UCAV air combat[C]//17th Asian Simulation Conference. Melaka,Malaysia: Springer, 2017: 274-286.
[10]	YANG Q M, ZHANG J D, SHI G Q, et al. Maneuver decision of UAV in short-range air combat based on deep reinforcement learning [J]. IEEE Access, 2020, 8:363-378.
[11]	LI Y T, HAN T, SUN C, et al. An optimization method of air combat situation assessment function based on inverse reinforcement learning [J]. Fire Control & Command Control, 2019, 44(8): 101-106 (in Chinese).
[12]	PETERS J, SCHAAL S. Natural actor-critic [J]. Neurocomputing,2008, 71(7/8/9): 1180-1190.
[13]	LV H W, GAO Y, HUANG Q L, et al. Research on multi-target assignment model in air combat [J]. Journal of Naval Aeronautical and Astronautical University,2008, 23(1): 59-61 (in Chinese).
[14]	JIANG L T, KOU Y N, WANG D, et al. A dynamic variable weight method for situation assessment in close-range air combat [J]. Electronic Optics & Control,2019, 26(4): 1-5 (in Chinese).
[15]	CHEN D J, WANG J. Air defense target threat assessment based on intuitionistic fuzzy sets [J]. Journal of Detection & Control, 2019, 41(4): 46-51 (in Chinese).
[16]	FAN Z H, SHI B H, CHEN J Y, et al. A novel dynamic bayesian network based threat assessment algorithm[C]//2017 4th International Conference on Systems and Informatics (ICSAI ). Hangzhou, China: IEEE,2017: 611-615.