基于目标检测和特征提取网络的运动无人机平台下多目标跟踪

doi:10.1007/s12204-022-2540-4

J Shanghai Jiaotong Univ Sci ›› 2024, Vol. 29 ›› Issue (3): 388-399.doi: 10.1007/s12204-022-2540-4

基于目标检测和特征提取网络的运动无人机平台下多目标跟踪

刘增敏1,2,3,4,6，王申涛5，姚莉秀1,2,3，蔡云泽1,2,3,4,6

（1.上海交通大学自动化系，上海200240；2. 系统控制与信息处理教育部重点实验室，上海200240；3. 上海工业智能管控工程技术研究中心，上海 200240；4. 海洋智能装备与系统教育部重点实验室，上海 200240；5. 京东企业业务增长部，北京100176；6. 上海交通大学海洋装备研究院，上海 200240）

接受日期:2021-10-10 出版日期:2024-05-28 发布日期:2024-05-28

Online Multi-Object Tracking Under Moving Unmanned Aerial Vehicle Platform Based on Object Detection and Feature Extraction Network

LIU Zengmin^1,2,3,4,6 (刘增敏), WANG Shentao⁵(王申涛)，YAO Lixiu^1,2,3 (姚莉秀),CAI Yunze^1,2,3,4,6∗(蔡云泽)

(1. Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, China; 2. Key Laboratory of System Control and Information Processing of Ministry of Education, Shanghai 200240, China; 3. Shanghai Engineering Research Center of Intelligent Control and Management, Shanghai 200240, China; 4. Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education, Shanghai Jiao Tong University, Shanghai 200240, China; 5. JD Business Growth Department, Beijing 100176, China; 6. Institute of Marine Equipment, Shanghai Jiao Tong University, Shanghai 200240, China)

Accepted:2021-10-10 Online:2024-05-28 Published:2024-05-28

摘要/Abstract

摘要： 为了解决无人机平台下小物体尺寸小、检测精度低的问题，基于深度聚合网络和高分辨率融合模块研究了一种目标检测算法。此外，还探索了一种目标检测与特征提取的联合网络，以构建实时多目标跟踪算法。针对无人机移动导致的目标关联失败问题，将图像配准应用于多目标跟踪，并提出了一种相机运动判别模型，以提高多目标跟踪算法的速度。仿真结果表明，提出的算法能够提高无人机平台下的多目标跟踪精度，并有效解决无人机移动导致的关联失败问题。

关键词: 移动无人机（UAV）平台, 小物体, 特征提取, 图像配准, 多目标跟踪

Abstract: In order to solve the problem of small object size and low detection accuracy under the unmanned aerial vehicle (UAV) platform, the object detection algorithm based on deep aggregation network and high-resolution fusion module is studied. Furthermore, a joint network of object detection and feature extraction is studied to construct a real-time multi-object tracking algorithm. For the problem of object association failure caused by UAV movement, image registration is applied to multi-object tracking and a camera motion discrimination model is proposed to improve the speed of the multi-object tracking algorithm. The simulation results show that the algorithm proposed in this study can improve the accuracy of multi-object tracking under the UAV platform, and effectively solve the problem of association failure caused by UAV movement.

中图分类号:

TP183

刘增敏1, 2, 3, 4, 6, 王申涛5, 姚莉秀1, 2, 3, 蔡云泽1, 2, 3, 4, 6. 基于目标检测和特征提取网络的运动无人机平台下多目标跟踪[J]. J Shanghai Jiaotong Univ Sci, 2024, 29(3): 388-399.

LIU Zengmin (刘增敏), WANG Shentao(王申涛), YAO Lixiu(姚莉秀), CAI Yunze(蔡云泽). Online Multi-Object Tracking Under Moving Unmanned Aerial Vehicle Platform Based on Object Detection and Feature Extraction Network[J]. J Shanghai Jiaotong Univ Sci, 2024, 29(3): 388-399.

参考文献

[1] CIAPARRONE G, LUQUE S′ANCHEZ F, TABIK S, et al. Deep learning in video multi-object tracking: A survey [J]. Neurocomputing, 2020, 381: 61-88.
[2] WANG C D. Key technologies of the real-time processing with low-altitude UAV video [D]. Wuhan: Wuhan University, 2018 (in Chinese).
[3] ZHANG X D, IZQUIERDO E, CHANDRAMOULI K. Dense and small object detection in UAV vision based on cascade network [C]//2019 IEEE/CVF International Conference on Computer Vision Workshop. Seoul: IEEE, 2019: 118-126.
[4] CAI Z W, VASCONCELOS N. Cascade R-CNN: Delving into high quality object detection [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018:6154-6162.
[5] CHEN C R, ZHANG Y, LV Q X, et al. RRNet: A hybrid detector for object detection in drone-captured images [C]//2019 IEEE/CVF International Conference on Computer Vision Workshop. Seoul: IEEE, 2019: 100-108.
[6] WANG W. Research on real-time vehicle detection and tracking algorithm based on UAV [D]. Qinhuangdao: Yanshan University, 2020 (in Chinese).
[7] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: Inverted residuals and linear bottlenecks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4510-4520.
[8] REDMON J, FARHADI A. YOLOv3: An incremental improvement [DB/OL]. (2018-04-08). https://arxiv.org/abs/1804.02767.
[9] ZHENG Z H, WANG P, LIU W, et al. Distance-IoU loss: Faster and better learning for bounding box regression [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12993-13000.
[10] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection [C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017:2999-3007.
[11] KIM M, ALLETTO S, RIGAZIO L. Similarity mapping with enhanced Siamese network for multi-object tracking [DB/OL]. (2016-09-28). https://arxiv.org/abs/1609.09156.
[12] WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric [C]//2017 IEEE International Conference on Image Processing. Beijing: IEEE, 2017: 3645-3649.
[13] BEWLEY A, GE Z Y, OTT L, et al. Simple online and realtime tracking [C]//2016 IEEE International Conference on Image Processing. Phoenix: IEEE, 2016: 3464-3468.
[14] MILAN A, REZATOFIGHI S H, DICK A, et al. Online multi-target tracking using recurrent neural networks [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2017, 31(1): 4225-4232.
[15] LI S Y, YEUNG D Y. Visual object tracking for unmanned aerial vehicles: A benchmark and new motion models [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2017, 31(1): 4140-4146.
[16] LOWE D G. Distinctive image features from scaleinvariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[17] PAN S Y, TONG Z H, ZHAO Y Y, et al. Multiobject tracking hierarchically in visual data taken from drones [C]//2019 IEEE/CVF International Conference on Computer Vision Workshop. Seoul: IEEE, 2019: 135-143.
[18] BERGMANN P, MEINHARDT T, LEAL-TAIX′E L. Tracking without bells and whistles [C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 941-951.
[19] VOIGTLAENDER P, KRAUSE M, OSEP A, et al. MOTS: Multi-object tracking and segmentation [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7934-7943.
[20] YU F, WANG D Q, SHELHAMER E, et al. Deep layer aggregation [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2403-2412.
[21] TIAN Z, SHEN C H, CHEN H, et al. FCOS: fully convolutional one-stage object detection [C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 9626-9635.
[22] MAHMOUDI N, AHADI S M, RAHMATI M. Multitarget tracking using CNN-based features: CNNMTT [J]. Multimedia Tools and Applications, 2019, 78(6): 7077-7096.
[23] PERNICI F, BARTOLI F, BRUNI M, et al. Memory based online learning of deep representations from video streams [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2324-2334.
[24] WANG Z D, ZHENG L, LIU Y X, et al. Towards real-time multi-object tracking [M]//Computer vision– ECCV 2020. Cham: Springer, 2020: 107-122.
[25] ZHANG Y F, WANG C Y, WANG X G, et al. FairMOT: On the fairness of detection and re-identification in multiple object tracking [J]. International Journal of Computer Vision, 2021, 129: 3069-3087.
[26] REZATOFIGHI H, TSOI N, GWAK J, et al. Generalized intersection over union: A metric and a loss for bounding box regression [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 658-666.
[27] MUNKRES J. Algorithms for the assignment and transportation problems [J]. Journal of the Society for Industrial and Applied Mathematics, 1957, 5(1): 32-38.
[28] BAY H, TUYTELAARS T, VAN GOOL L. SURF: Speeded up robust features [M]//Computer vision–ECCV 2006. Berlin, Heidelberg: Springer, 2006: 404-417.
[29] CALONDER M, LEPETIT V, STRECHA C, et al. BRIEF: Binary robust independent elementary features [M]//Computer vision – ECCV 2010. Berlin, Heidelberg: Springer, 2010: 778-792.
[30] FISCHLER M, BOLLES R. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography [J]. Communications of the ACM, 1981, 24: 381-395.
[31] ZHOU X, KOLTUN V, KR¨AHENB¨UHL P. Tracking objects as points [M]//Computer vision – ECCV 2020. Cham, Springer, 2020: 474-490.
[32] ZHOU X Y, WANG D Q, KR¨AHENB¨UHL P. Objects as points [DB/OL]. (2019-04-16). https://arxiv.org/abs/1904.07850.

基于目标检测和特征提取网络的运动无人机平台下多目标跟踪

Online Multi-Object Tracking Under Moving Unmanned Aerial Vehicle Platform Based on Object Detection and Feature Extraction Network

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	曾志贤，曹建军，翁年凤，袁震，余旭. 基于细粒度联合注意力机制的图像-文本跨模态实体分辨[J]. J Shanghai Jiaotong Univ Sci, 2023, 28(6): 728-737.
[2]	詹燕, 陈志慧, 朱宝昌, 朱婷婷, 邵益平, 鲁建厦. 基于自适应颜色快速点特征直方图的托盘识别方法[J]. 上海交通大学学报, 2023, 57(3): 297-308.
[3]	. 基于多视角道路相机的时空关联三维车辆检测与跟踪系统[J]. J Shanghai Jiaotong Univ Sci, 2023, 28(1): 52-60.
[4]	张晏合, 臧月进, 陈渤, 徐铭晟. 基于解耦表征变分自编码机的雷达目标识别算法[J]. 空天防御, 2022, 5(2): 87-93.
[5]	何子述, 苏洋, 程婷. 基于共址MIMO雷达组网系统的机会协同目标跟踪方法[J]. 空天防御, 2022, 5(1): 6-11.
[6]	王微, 王冰, 胡雄, 孙德建. 基于在线改进符号序列熵与逻辑回归模型的岸桥起升减速箱在线退化评估[J]. 上海交通大学学报, 2021, 55(10): 1272-1280.
[7]	王俊, 王赛, 任俞明, 陈德红, 崔闪, 魏少明. 结合深度学习去噪和超分辨的SAR检测识别[J]. 空天防御, 2020, 3(3): 24-30.
[8]	颜波，张磊，褚学宁. 基于卷积神经网络的用户感知评估建模[J]. 上海交通大学学报, 2019, 53(7): 844-851.
[9]	商诺诺, 梁艳, 翟恒峰, 原浩娟. 一种高效的雷达信号全脉冲特征提取方法[J]. 空天防御, 2018, 1(4): 37-43.
[10]	吴倩红，韩蓓，冯琳，李国杰，江秀臣. “人工智能+”时代下的智能电网预测分析[J]. 上海交通大学学报（自然版）, 2018, 52(10): 1206-1219.
[11]	王瑞，刘宾，周天润，杨羽. 基于协同表示的声振传感器网络车辆分类识别[J]. 上海交通大学学报（自然版）, 2018, 52(1): 103-110.
[12]	王嗣阳，许黎明，赖小平. 砂轮磨削过载判据及其快速诊断[J]. 上海交通大学学报（自然版）, 2015, 49(09): 1346-1352.
[13]	成谢锋,傅女婷. 心音身份识别综述[J]. 上海交通大学学报（自然版）, 2014, 48(12): 1745-1750.
[14]	王佳婧1，2，张树生1，何卫平1. 基于多信息融合的失效DPM码识别[J]. 上海交通大学学报（自然版）, 2014, 48(12): 1675-1680.
[15]	余焕伟，叶震，张志芬，陈华斌，陈善本. Al-Mg合金钨极气体保护焊接电弧的光谱特征提取方法[J]. 上海交通大学学报（自然版）, 2013, 47(11): 1655-1660.