Intercept Guidance Law with a Low Acceleration Ratio Based on Hierarchical Reinforcement Learning
WANG Xu1， CAI Yuanli1， ZHANG Xuecheng2，
ZHANG Rongliang3， HAN Chenglong3
1. Faculty of Electronic and Information Engineering， Xi’an Jiaotong University， Xi’an 710049， Shaanxi， China；
2. Third Military Representative Office of Army Equipment Department in Shanghai， Shanghai 200031, China；
3. Shanghai Electro-Mechanical Engineering Institute， Shanghai 201109，China
Abstract This paper has proposed an intercept guidance law based on hierarchical reinforcement learning to solve the three-dimensional maneuvering target intercept guidance problem with constraints of low acceleration ratio and bearings-only measurement. The aforementioned problem was initially modelled using a Markov decision process model, where a heuristic reward function was applied considering both the energy consumption and the missile-to-target line of sight (LOS) angular rate. Besides, the policy of two levels was built up with the lower-level policy generating the required guidance command and being supervised by subgoals that were instructed by the higher levels, allowing the convergence of the LOS angular rate and guaranteeing the successful interception against a maneuvering target. Simulation results have validated the superiority of the proposed method compared with the augmented proportional navigation guidance law in terms of intercept accuracy and hit probability, and its required acceleration ratio is much lower.
2024, Vol. 7 
