Landing State Recognition of Carrier-Based Aircraft Based on Adaptive Feature Enhancement and Fusion

doi:10.16183/j.cnki.jsjtu.2023.263

Abstract

Abstract:

The recognition of engagement state aids landing signal officers in formulating command decisions promptly and precisely, which is crucial for guiding carrier-based aircraft landings. A method is proposed for recognizing the engagement state, leveraging adaptive feature enhancement and fusion, which includes an attention mechanism-based feature enhancement module and a multi-scale feature fusion module. The front module enhances visual representation by segmenting feature maps and concatenating spatial and channel domains, and the back module merges high-resolution shallow features with semantically rich deep features to fully utilize contextual information. A prototype system is developed to recognize landing engagement states based on the wearable augmented reality devices. To evaluate the performance of the method proposed, hybrid datasets of landing operations are constructed. The results show that the proposed algorithm outperforms baseline algorithms and meets the application requirements of engagement state recognition.

Key words: carrier-based aircraft, arrested landing, feature fusion, attention mechanism, state recognition

CLC Number:

TP391.41

WANG Ke, LIU Yiyang, YANG Jie, LU Aiguo, LI Zhe, XU Mingliang. Landing State Recognition of Carrier-Based Aircraft Based on Adaptive Feature Enhancement and Fusion[J]. Journal of Shanghai Jiao Tong University, 2025, 59(2): 274-282.

Figures/Tables 16

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Tab.1

Fig.6

Fig.7

Tab.2

Fig.8

Tab.3

Tab.4

Fig.9

Tab.5

Tab.6

Fig.10

References 18

[1]	王可, 徐明亮, 李亚飞, 等. 一种面向航空母舰甲板运动状态预估的鲁棒学习模型[J]. 自动化学报, 2024, 50(9): 1785-1793.
	WANG Ke, XU Mingliang, LI Yafei, et al. A robust learning model for deck motion prediction of aircraft carrier[J]. Acta Automatica Sinica, 2024, 50(9): 1785-1793.
[2]	李亚飞, 吴庆顺, 徐明亮, 等. 基于强化学习的舰载机保障作业实时调度方法[J]. 中国科学: 信息科学, 2021, 51(2): 247-262.
	LI Yafei, WU Qingshun, XU Mingliang, et al. Real-time scheduling for carrier-borne aircraft support operations: A reinforcement learning approach[J]. Scientia Sinica (Informationis), 2021, 51(2): 247-262.
[3]	薛均晓, 徐明亮, 李亚飞, 等. 面向航空母舰电子显灵板的多智能体建模技术研究进展[J]. 计算机辅助设计与图形学学报, 2021, 33(10): 1475-1485.
	XUE Junxiao, XU Mingliang, LI Yafei, et al. Research progress of multi-agent technology for aircraft carrier electronic display panel[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(10): 1475-1485.
[4]	王华, 韩璐, 楚世理, 等. 基于Frenet标架下三维元胞自动机的航母舰载机集群运动建模[J]. 计算机辅助设计与图形学学报, 2018, 30(9): 1719-1727.
	WANG Hua, HAN Lu, CHU Shili, et al. Shipboard aircraft swarm modeling using a 3D cellular automata model under the frenet frame[J]. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(9): 1719-1727.
[5]	江驹, 王新华, 甄子洋. 舰载机起飞着舰引导与控制[M]. 北京: 科学出版社, 2019.
	JIANG Ju, WANG Xinhua, ZHEN Ziyang. Guidance and control of carrier-based aircraft taking off and landing[M]. Beijing: Science Press, 2019.
[6]	薛均晓, 孔祥燕, 郭毅博, 等. 基于深度强化学习的舰载机动态避障方法[J]. 计算机辅助设计与图形学学报, 2021, 33(7): 1102-1112.
	XUE Junxiao, KONG Xiangyan, GUO Yibo, et al. Dynamic obstacle avoidance method for carrier aircraft based on deep reinforcement learning[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(7): 1102-1112.
[7]	汪丁, 黄葵, 朱兴动, 等. 基于改进YOLOv4-tiny的舰面多目标检测算法[J]. 兵工自动化, 2022, 41(10): 1-6.
	WANG Ding, HUANG Kui, ZHU Xingdong, et al. Multi-target detection algorithm for ship surface based on improved YOLOv4-tiny[J]. Ordnance Industry Automation, 2022, 41(10): 1-6.
[8]	范加利, 田少兵, 黄葵, 等. 基于Faster R-CNN的航母舰面多尺度目标检测算法[J]. 系统工程与电子技术, 2022, 44(1): 40-46. doi: 10.12305/j.issn.1001-506X.2022.01.06
	FAN Jiali, TIAN Shaobing, HUANG Kui, et al. Multi-scale object detection algorithm for aircraft carrier surface based on Faster R-CNN[J]. Systems Engineering and Electronics, 2022, 44(1): 40-46. doi: 10.12305/j.issn.1001-506X.2022.01.06
[9]	朱兴动, 汪丁, 范加利, 等. 复杂场景下基于增强YOLOv3的舰面多目标检测[J]. 计算机工程与应用, 2022, 58(13): 177-184. doi: 10.3778/j.issn.1002-8331.2012-0411
	ZHU Xingdong, WANG Ding, FAN Jiali, et al. Multitarget detection based on enhanced YOLOv3 in complex scenarios[J]. Computer Engineering and Applications, 2022, 58(13): 177-184. doi: 10.3778/j.issn.1002-8331.2012-0411
[10]	XU P C, GUO Z Y, LIANG L, et al. MSF-net: Multi-scale feature learning network for classification of surface defects of multifarious sizes[J]. Sensors, 2021, 21(15): 5125.
[11]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 936-944.
[12]	ZHANG S F, CHI C, YAO Y Q, et al. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 9756-9765.
[13]	GUO D Z, ZHU L G, LU Y H, et al. Small object sensitive segmentation of urban street scene with spatial adjacency between object classes[J]. IEEE Transactions on Image Processing, 2018: 28(6): 2643-2653.
[14]	谢磊, 丁达理, 魏政磊, 等. AdaBoost-PSO-LSTM网络实时预测机动轨迹[J]. 系统工程与电子技术, 2021, 43(6): 1651-1658. doi: 10.12305/j.issn.1001-506X.2021.06.23
	XIE Lei, DING Dali, WEI Zhenglei, et al. Real time prediction of maneuver trajectory for AdaBoost-PSO-LSTM network[J]. Systems Engineering and Electronics, 2021, 43(6): 1651-1658. doi: 10.12305/j.issn.1001-506X.2021.06.23
[15]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot MultiBox detector[M]//Computer Vision-ECCV 2016. Cham: Springer, 2016: 21-37.
[16]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 2999-3007.
[17]	REDMON J, FARHADI A. YOLOv3: An incremental improvement[DB/OL]. (2018-04-08)[2022-06-09]. https://arxiv.org/abs/1804.02767.pdf.
[18]	FU C Y, LIU W, RANGA A, et al. DSSD: Deconvolutional single shot detector[DB/OL]. (2017-01-23)[2022-07-10]. https://arxiv.org/abs/1701.06659.pdf.

因素	类别	样本数量
气象条件	晴天	1600
	阴天	1000
	雨天	1000
光照条件	亮场	2300
	暗场	1300

因素	类别	样本数量
气象条件	晴天	2400
	阴天	1000
	雨天	800
光照条件	亮场	2200
	暗场	2000

方法	分类			检测
方法	P/%	R/%	F/%	mAP/%	FPS/(帧·s^-1)
SSD300	77.8	33.2	46.5	56.6	57.1
SSD512	93.2	27.2	42.1	62.2	31.2
RetinaNet	67.7	79.5	73.1	65.2	33.2
YOLOv3	67.2	80.5	73.3	74.2	41.2
DSSD	84.2	53.8	65.6	73.5	19.7
本文方法	84.4	81.1	82.4	76.8	26.8

方法	分类			检测
方法	P/%	R/%	F/%	mAP/%	FPS/(帧·s^-1)
SSD300	67.2	22.2	33.4	54.1	57.1
SSD512	66.3	38.4	42.5	58.5	31.2
RetinaNet	61.8	60.5	61.1	59.3	33.2
YOLOv3	61.1	71.4	65.8	61.4	41.2
DSSD	64.2	32.2	42.9	57.1	19.7
本文方法	72.4	71.9	72.3	64.2	26.8

方法	平均精度/%
方法	鸟	瓶子	船	椅子
SSD300	70.3	45.3	63.2	54.3
SSD512	74.4	50.1	70.3	57.1
RetinaNet	75.8	54.8	71.5	59.2
YOLOv3	78.3	57.2	70.9	60.2
DSSD	79.8	53.9	67.5	58.2
本文	80.3	59.2	75.4	61.5