Airfield Multi-Scale Object Detection for Visual Navigation in Civil Aircraft

doi:10.16183/j.cnki.jsjtu.2024.206

Abstract

Abstract:

The visual assistance driving system for civil aviation aircraft captures information about the surrounding threat scenario using airborne visual sensors, providing pilots with additional information to aid decision-making. However, the threat objects in the airfield obtained by the optical sensors on the airborne differ significantly in scale, and the computing capacity of the onboard platform is limited. Current methods for object detection do not meet the requirements for visual assistance in driving scenarios. To address this issue, a lightweight multi-scale object detection algorithm based on YOLOv5s is proposed. First, the CA-BIFPN feature fusion network is designed by combining the weighted bidirectional feature pyramid network (BIFPN) with the coordinate attention (CA) attention mechanism, which aims to enhance the feature expression of small objects and to improve the capacity of the model to learn multi-scale objects. Then, the GSConv decoupled detection head is designed to improve object detection accuracy by making classification and regression independent. To enhance the detection speed of the network and enable real-time detection of airfield objects, a cross-level partial lightweight neck module is designed to reduce the additional parameters introduced by the decoupled head. A self-built multi-scale airfield object dataset containing real-world and simulated data from airborne visual sensors from a civil aviation aircraft perspective is established to verify the performance of the proposed algorithm. The experiments conducted on this dataset demonstrate that the detection accuracy of the proposed algorithm surpasses that of faster R-CNN, SSD, and other classic multi-scale object detection algorithms like YOLOv6, YOLOv7, and YOLOX. The achieved mean average precision is 71.40%, which is 4.19% higher than that of YOLOv5s. Furthermore, the detection frame rate achieves 71 frame per second on the simulated airborne computing platform, which satisfies the real-time detection requirements.

Key words: YOLOv5s algorithm, civil aircraft visual navigation, multi-scale object detection, feature fusion network, decoupled detection head

CLC Number:

ZHANG Tao, ZHANG Xuerui, CHEN Yong, ZHONG Kelin, LUO Qijun. Airfield Multi-Scale Object Detection for Visual Navigation in Civil Aircraft[J]. Journal of Shanghai Jiao Tong University, 2024, 58(11): 1816-1825.

Figures/Tables 13

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Tab.1

Tab.2

Tab.3

Tab.4

References 25

[1]	Airbus Pioneering Sustainable Aerospace. Airbus demonstrates the first fully automatic vision-based take-off[EB/OL]. (2020-01-16)[2024-05-20]. https://www.airbus.com/en/newsroom/press-releases/2020-01-airbus-demonstrates-first-fully-automatic-vision-based-take-off..
[2]	陈科圻, 朱志亮, 邓小明, 等. 多尺度目标检测的深度学习研究综述[J]. 软件学报, 2021, 32(4): 1201-1227.
	CHEN Keqi, ZHU Zhiliang, DENG Xiaoming, et al. Deep learning for multi-scale object detection: A survey[J]. Journal of Software, 2021, 32(4): 1201-1227.
[3]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031 pmid: 27295650
[4]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]//European Conference on Computer Vision. Cham, Germany: Springer, 2016: 21-37.
[5]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 936-944.
[6]	TAN M X, PANG R M, LE Q V. EfficientDet: Scalable and efficient object detection[C]//2020 IEEE Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 10778-10787.
[7]	REDMON J, FARHADI A. YOLOv3: An incremental improvement[DB/OL]. (2018-04-08) [2024-05-20]. http://arxiv.org/abs/1804.02767.
[8]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2020, 42(2): 318-327.
[9]	韩松臣, 张比浩, 李炜, 等. 基于改进Faster-RCNN的机场场面小目标物体检测算法[J]. 南京航空航天大学学报, 2019, 51(6): 735-741.
	HAN Songchen, ZHANG Bihao, LI Wei, et al. Small target detection in airport scene via modified faster-RCNN[J]. Journal of Nanjing University of Aeronautics & Astronautics, 2019, 51(6): 735-741.
[10]	黄国新, 李炜, 张比浩, 等. 改进SSD的机场场面多尺度目标检测算法[J]. 计算机工程与应用, 2022, 58(5): 264-270. doi: 10.3778/j.issn.1002-8331.2010-0110
	HUANG Guoxin, LI Wei, ZHANG Bihao, et al. Improved SSD-based multi-scale object detection algorithm in airport surface[J]. Computer Engineering and Applications, 2022, 58(5): 264-270. doi: 10.3778/j.issn.1002-8331.2010-0110
[11]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016: 770-778.
[12]	LI H L, LI J, WEI H B, et al. Slim-neck by GSConv: A lightweight-design for real-time detector architectures[J]. Journal of Real-Time Image Processing, 2024, 21(3): 62.
[13]	CHOLLET F. Xception: Deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 1800-1807.
[14]	TIAN Y L, ZHANG Q S, REN Z L, et al. Multi-scale dilated convolution network based depth estimation in intelligent transportation systems[J]. IEEE Access, 2019, 7: 185179-185188.
[15]	ZHANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: An extremely efficient convolutional neural network for mobile devices[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 6848-6856.
[16]	CHIU Y C, TSAI C Y, RUAN M D, et al. Mobilenet-SSDv2: An improved object detection model for embedded systems[C]//2020 International Conference on System Science and Engineering. Kagawa, Japan: IEEE, 2020: 1-5.
[17]	BOCHKOVSKIY A, WANG C Y, LIAO H Y. YOLOv4: Optimal speed and accuracy of object detection[DB/OL]. (2020-04-23) [2024-06-15]. https://arxiv.org/abs/2004.10934v1.
[18]	Ultralytics. YOLOv5[EB/OL]. (2020-06-03) [2024-06-15]. https://github.com/ultralytics/yolov5.
[19]	YAN K, HUA M, LI Y L. Multi-target detection in airport scene based on Yolov5[C]//2021 IEEE 3rd International Conference on Civil Aviation Safety and Information Technology. Changsha, China: IEEE, 2021: 1175-1177.
[20]	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 8759-8768.
[21]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: Common objects in context[C]//European Conference on Computer Vision. Cham, Germany: Springer, 2014: 740-755.
[22]	GUPTA C, GILL N S, GULIA P, et al. A novel finetuned YOLOv6 transfer learning model for real-time object detection[J]. Journal of Real-Time Image Processing, 2023, 20(3): 42.
[23]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//2023 IEEE Conference on Computer Vision and Pattern Recognition. Vancouver, Canada: IEEE, 2023: 7464-7475.
[24]	SAFALDIN M, ZAGHDEN N, MEJDOUB M. An improved YOLOv8 to detect moving objects[J]. IEEE Access, 2024, 12: 59782-59806.
[25]	DAI Z Y. Uncertainty-aware accurate insulator fault detection based on an improved YOLOX model[J]. Energy Reports, 2022, 8: 12809-12821.

名称	mAP/%	参数量/MB	计算量/GB	FPS/(帧·s^-1)
YOLOv5s	67.21	7.1	15.8	84
YOLOv5s-D	69.10	14.3	56.2	31
YOLOv5s-GSD	69.00	11.2	45.3	47

名称	mAP/%	参数量/MB	计算量/GB	FPS/ (帧·s^-1)
YOLOv5s-GSD	69.0	11.2	45.3	47
YOLOv5s-GSD-SN	68.2	9.7	33.7	71
YOLOv5s-GSD-DCSN	68.8	9.7	33.7	71

方法	mAP/%	参数量/MB	计算量/GB	FPS/ (帧·s^-1)
YOLOv5s	67.21	7.1	15.8	84
+CA-BIFPN	68.49	7.2	16.2	75
+GSDetector	70.95	12.3	43.7	47
+DC Slim-Neck	70.93	9.8	30.1	71
+EIoU	71.40	9.8	30.2	71

算法名称	mAP/%	参数量/ MB	计算量/ GB	FPS/ (帧·s^-1)
RetinaNet^[8]	63.28	8.7	28.0	12
Faster R-CNN^[3]	68.95	28.3	947.3	15
SSD^[4]	64.82	23.6	235.2	49
YOLOv5s^[18]	67.21	7.1	15.8	84
YOLOv6^[22]	64.53	18.5	45.2	45
YOLOv7^[23]	66.14	37.2	105.7	40
YOLOv8^[24]	66.96	15.7	39.3	49
YOLOX^[25]	67.05	11.5	38.9	42
本文算法	71.40	9.8	30.2	71