Missile-Target Situation Assessment Model Based on Reinforcement Learning

doi:10.1007/s12204-020-2226-8

Journal of Shanghai Jiao Tong University(Science) ›› 2020, Vol. 25 ›› Issue (5): 561-568.doi: 10.1007/s12204-020-2226-8

Missile-Target Situation Assessment Model Based on Reinforcement Learning

ZHANG Yun (张贇), Lü Runyan (吕润妍), CAI Yunze (蔡云泽)

(Department of Automation; Key Laboratory of System Control and Information Processing of Ministry of Education;
Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education,
Shanghai Jiao Tong University, Shanghai 200240, China)

出版日期:2020-10-28 发布日期:2020-09-11
通讯作者: CAI Yunze (蔡云泽) E-mail:yzcai@sjtu.edu.cn

Missile-Target Situation Assessment Model Based on Reinforcement Learning

ZHANG Yun (张贇), Lü Runyan (吕润妍), CAI Yunze (蔡云泽)

(Department of Automation; Key Laboratory of System Control and Information Processing of Ministry of Education;
Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education,
Shanghai Jiao Tong University, Shanghai 200240, China)

Online:2020-10-28 Published:2020-09-11
Contact: CAI Yunze (蔡云泽) E-mail:yzcai@sjtu.edu.cn

摘要/Abstract

摘要： In situation assessment (SA) of missile versus target fighter, the traditional SA models generally
have the characteristics of strong subjectivity and poor dynamic adaptability. This paper considers SA as an
expectation of future returns and establishes a missile-target simulation battle model. The actor-critic (AC)
algorithm in reinforcement learning (RL) is used to train the evaluation network, and a missile-target SA model
is established in simulation battle training. Simulation and comparative experiments show that the model can
effectively estimate the expected effect of missile attack under the current situation, and it provides an effective
basis for missile attack decision.

关键词: situation assessment (SA), battle model, reinforcement learning (RL), actor-critic (AC) algorithm

Abstract: In situation assessment (SA) of missile versus target fighter, the traditional SA models generally
have the characteristics of strong subjectivity and poor dynamic adaptability. This paper considers SA as an
expectation of future returns and establishes a missile-target simulation battle model. The actor-critic (AC)
algorithm in reinforcement learning (RL) is used to train the evaluation network, and a missile-target SA model
is established in simulation battle training. Simulation and comparative experiments show that the model can
effectively estimate the expected effect of missile attack under the current situation, and it provides an effective
basis for missile attack decision.

Key words: situation assessment (SA), battle model, reinforcement learning (RL), actor-critic (AC) algorithm

中图分类号:

TP 391
TP 18

ZHANG Yun, Lü Runyan, CAI Yunze . Missile-Target Situation Assessment Model Based on Reinforcement Learning[J]. Journal of Shanghai Jiao Tong University(Science), 2020, 25(5): 561-568.

参考文献 16

[1]	ENDSLEY M R. Toward a theory of situation awareness in dynamic systems [J]. Human Factors, 1995,37(1): 32-64.
[2]	STEINBERG A N, BOWMAN C L,WHITE F E. Revisions to the JDL data fusion model [J]. Proceedings of SPIE, 1999, 3719: 430-441.
[3]	CHEN X, WEI X M, XU G Y. Multiple unmanned aerial vehicle decentralized cooperative air combat decision making with fuzzy situation [J]. Journal of Shanghai Jiao Tong University, 2014, 48(7): 907-913(in Chinese).
[4]	YAN C C, HAO Y S. Threat assessment of aerial target based on AHP [J]. Computing Technology and Automation,2011, 30(2): 118-121 (in Chinese).
[5]	CHEN J, YU G H, GAO X G. Cooperative threat assessment of multi-aircrafts based on synthetic fuzzy cognitive map [J]. Journal of Shanghai Jiao Tong University(Science), 2012, 17(2): 228-232.
[6]	LU C G, ZHOU Z L, LIU H Q, et al. Situation assessment of far-distance attack air combat based on mixed dynamic bayesian networks [C]//Proceedings of the 37th Chinese Control Conference. Wuhan, China:Chinese Association of Automation, 2018: 1133-1138.
[7]	PENG P, WEN Y, YANG Y D, et al. Multiagent bidirectionally-coordinated nets: Emergence of human-level coordination in learning to play Star-Craft combat games [EB/OL]. (2017-09-14) [2020-07-14]. https://arxiv.org/pdf/1703.10069v4.pdf.
[8]	ZHOU Z Q, QIAN J G, WANG Y Z. Research on ballistic missile situation grade model based on BP neural network [J]. Fire Control & Command Control, 2015,40(5): 53-56 (in Chinese).
[9]	LIU P, MA Y F. A deep reinforcement learning based intelligent decision method for UCAV air combat[C]//17th Asian Simulation Conference. Melaka,Malaysia: Springer, 2017: 274-286.
[10]	YANG Q M, ZHANG J D, SHI G Q, et al. Maneuver decision of UAV in short-range air combat based on deep reinforcement learning [J]. IEEE Access, 2020, 8:363-378.
[11]	LI Y T, HAN T, SUN C, et al. An optimization method of air combat situation assessment function based on inverse reinforcement learning [J]. Fire Control & Command Control, 2019, 44(8): 101-106 (in Chinese).
[12]	PETERS J, SCHAAL S. Natural actor-critic [J]. Neurocomputing,2008, 71(7/8/9): 1180-1190.
[13]	LV H W, GAO Y, HUANG Q L, et al. Research on multi-target assignment model in air combat [J]. Journal of Naval Aeronautical and Astronautical University,2008, 23(1): 59-61 (in Chinese).
[14]	JIANG L T, KOU Y N, WANG D, et al. A dynamic variable weight method for situation assessment in close-range air combat [J]. Electronic Optics & Control,2019, 26(4): 1-5 (in Chinese).
[15]	CHEN D J, WANG J. Air defense target threat assessment based on intuitionistic fuzzy sets [J]. Journal of Detection & Control, 2019, 41(4): 46-51 (in Chinese).
[16]	FAN Z H, SHI B H, CHEN J Y, et al. A novel dynamic bayesian network based threat assessment algorithm[C]//2017 4th International Conference on Systems and Informatics (ICSAI ). Hangzhou, China: IEEE,2017: 611-615.

相关文章 15

[1]	. [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(6): 757-767.
[2]	. [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(2): 190-201.
[3]	. [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(2): 240-249.
[4]	. [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 7-14.
[5]	. [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 24-35.
[6]	. [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 99-111.
[7]	. [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 121-136.
[8]	. [J]. J Shanghai Jiaotong Univ Sci, 2021, 26(5): 577-586.
[9]	. [J]. J Shanghai Jiaotong Univ Sci, 2021, 26(5): 587-597.
[10]	. [J]. J Shanghai Jiaotong Univ Sci, 2021, 26(5): 670-679.
[11]	SHI Lianxing (石连星), WANG Zhiheng (王志恒), LI Xiaoyong (李小勇) . Novel Data Placement Algorithm for Distributed Storage System Based on Fault-Tolerant Domain[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(4): 463-470.
[12]	ZHAN Zhu (占竹), ZHANG Wenjun (张文俊), CHEN Xia (陈霞), WANG Jun (汪军) . Objective Evaluation of Fabric Flatness Grade Based on Convolutional Neural Network[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(4): 503-510.
[13]	LIU Ziwen (刘子文), XIAO Lei (肖雷), BAO Jinsong (鲍劲松), TAO Qingbao (陶清宝) . Bearing Incipient Fault Detection Method Based on Stochastic Resonance with Triple-Well Potential System[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(4): 482-487.
[14]	MA Qunsheng (马群圣), CEN Xingxing (岑星星), YUAN Junyi (袁骏毅), HOU Xumin (侯旭敏). Word Embedding Bootstrapped Deep Active Learning Method to Information Extraction on Chinese Electronic Medical Record[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(4): 494-502.
[15]	SHAN Rui (山蕊), JIANG Lin (蒋林), WU Haoyue (吴昊玥), HE Feilong (贺飞龙), LIU Xinchuang (刘新闯). Dynamical Self-Reconfigurable Mechanism for Data-Driven Cell Array[J]. J Shanghai Jiaotong Univ Sci, 2021, 26(4): 511-521.

Missile-Target Situation Assessment Model Based on Reinforcement Learning

Missile-Target Situation Assessment Model Based on Reinforcement Learning

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献 16

相关文章 15

编辑推荐

Metrics

本文评价