Simultaneous Detection and Localization for Intelligent Vehicles Based on HD Map Matching and Semantic Likelihood Model

doi:10.16183/j.cnki.jsjtu.2023.086

Abstract

Abstract:

Accurate matching between in-vehicle sensor data and high-definition (HD) maps is crucial to improve the performance of perception and localization of intelligent vehicles. A novel algorithm of HD map matching based on the developed semantic likelihood model (SLM) is proposed to achieve intelligent vehicle localization and object detection simultaneously. First, semantic pavement objects are extracted from front-view images by using U-Net, and SLM is constructed with kernel density estimation (KDE). Under a particle filter framework, the likelihood between the sensor data and HD map is calculated by projecting each sample point from HD map with pose transformation onto SLM to update the weight of each particle. Simultaneously, accurate detection of pavement markings is accomplished by projecting all elements onto the HD map with the computed localization results. In the experiment, data collected on campus and on urban roads are used to validate the proposed algorithm. The experimental results show that the localization errors in both scenarios are about 14 cm, and the mean intersection over union (MIoU) of road marking detection is above 80. The results demonstrate that the proposed algorithm can significantly improve both localization and detection performance by effectively utilizing the prior information of HD maps, compared with the state of the art (SOTA) methods, such as deep learning-based detection methods.

Key words: intelligent vehicle localization, high-definition (HD) map, semantic likelihood model (SLM), particle filter

CLC Number:

TP242.6

LAI Guoliang, HU Zhaozheng, ZHOU Zhe, WAN Jinjie, REN Jingyuan. Simultaneous Detection and Localization for Intelligent Vehicles Based on HD Map Matching and Semantic Likelihood Model[J]. Journal of Shanghai Jiao Tong University, 2024, 58(10): 1618-1628.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Tab.1

Fig.8

Fig.9

Tab.2

Fig.10

Tab.3

Fig.11

References 21

[1]	JO H G, LEE W, KIM E. Mixture Density-PoseNet and its application to monocular camera-based global localization[J]. IEEE Transactions on Industrial Informatics, 2021, 17(1): 388-397.
[2]	MA Y F, WANG Z Y, YANG H, et al. Artificial intelligence applications in the development of autonomous vehicles: A survey[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 7(2): 315-329.
[3]	中国科学技术协会. 中科协发布2020重大科学问题和工程技术难题[EB/OL]. (2020-08-16)[2023-03-09]. https://www.cast.org.cn/art/2020/8/16/art_90_130822.html.
	China Association for Science and Technology. 2020 CAST releases 2020 major scientific and engineeringtechnical challenges[EB/OL]. (2020-08-16)[2023-03-09]. https://www.cast.org.cn/art/2020/8/16/art_90_130822.html.
[4]	ZHONG H X, WANG R. A visual-degradation-inspired model with HSV color-encoding for contour detection[J]. Journal of Neuroscience Methods, 2022, 369(1): 1-12.
[5]	BEN C, BUSHNAG A. Color edge detection using multidirectional sobel filter and fuzzy fusion[J]. Computers, Materials & Continua, 2023, 2(2): 2839-2852.
[6]	NEVEN D, DE B, GEORGOULIS S, et al. Towards end-to-end lane detection: An instance segmentation approach[C]// IEEE Intelligent Vehicles Symposium (IV). Jiangsu, China: IEEE, 2018: 286-291.
[7]	JANG W, HYUN J, AN J, et al. A lane-level road marking map using a monocular camera[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(1): 187-204.
[8]	CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al. ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6): 1874-1890.
[9]	LIU R, WANG J L, ZHANG B Q. High definition map for automated driving: Overview and analysis[J]. Journal of Navigation, 2020, 73(2): 324-341.
[10]	胡兵, 杨明, 郭林栋, 等. 基于地面快速鲁棒特征的智能车全局定位方法[J]. 上海交通大学学报, 2019, 53(2): 203-208. doi: 10.16183/j.cnki.jsjtu.2019.02.011
	HU Bing, YANG Ming, GUO Lindong, et al. Global localization for intelligent vehicle using ground SURF[J]. Journal of Shanghai Jiao Tong University, 2019, 53(2): 203-208.
[11]	CUI D, XUE J, ZHENG N. Real-time global localization of robotic cars in lane level via lane marking detection and shape registration[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 17(4): 1039-1050.
[12]	王相龙, 胡钊政, 李祎承, 等. 基于路面指纹的高精度车辆定位[J]. 交通运输系统工程与信息, 2018, 18(4): 38-45.
	WANG Xianglong, HU Zhaozheng, LI Yicheng, et al. High accuracy vehicle localization by referring to pavement fingerprint[J]. Journal of Transportation Systems Engineering and Information Technology, 2018, 18(4): 38-45.
[13]	LIN C, LIAN F L. System integration of sensor-fusion localization tasks using vision-based driving lane detection and road-marker recognition[J]. IEEE Systems Journal, 2020, 14(3): 4523-4534.
[14]	XIAO Z Y, YANG D G, WEN T P, et al. Monocular localization with Vector HD Map (MLVHM): A low-cost method for commercial IVs[J]. Sensors, 2020, 20(7): 1870-1-24.
[15]	RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-assisted Intervention. Munich, Germany: Springer, 2015: 234-241.
[16]	ASAM. ASAM OpenDRIVE[EB/OL]. (2020-11-14)[2023-03-09]. https://www.asam.net/standards/detail/opendrive/.
[17]	THRUN S. Probabilistic robotics[J]. Communications of the ACM, 2002, 45(3): 52-57.
[18]	国家市场监督管理总局, 国家标准化管理委员会. 道路交通标志和标线. 第2部分:道路交通标志. GB 5768.2—2022[S]. 北京: 中国标准出版社, 2022.
	State Administration for Market Regulation, Standardization Administration of China. Road traffic signs and markings. Part 2:Road traffic signs. GB 5768.2—2022[S]. Beijing: Standards Press of China, 2022.
[19]	ZHANG Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.
[20]	CAO H, WANG Y, CHENG J, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation[C]// European Conference on Computer Vision. Cham, Switzerland: Springer Nature Switzerland, 2022: 205-218.
[21]	LIN S C, LEE H W, HSIEH Y H, et al. Masked attention ConvNeXt Unet with Multi-Synthesis dynamic weighting for anomaly detection and localization[C]// British Machine Vision Conference. Aberdeen, England: Machine Translation Springer, 2023: 722-735.

方法	最小误差/cm	最大误差/cm	平均误差/cm
本文算法	3.4	49.9	14.4
文献[11]	11.0	125.8	54.7
文献[13]	11.8	59.5	30.4

方法	最小误差/cm	最大误差/cm	平均误差/cm
本文算法	2.8	26.9	14.0
文献[11]	2.2	103.8	45.9
文献[13]	17.2	29.8	24.2

类别	IoU/%
类别	本文方法	Swin- Unet^[20]	ConvNext- Unet^[21]	U-Net
虚线	86.0	74.0	65.3	70.1
白色实线	90.6	65.7	56.3	78.1
停止线	85.7	69.9	64.8	62.3
直行箭头	87.6	72.6	71.8	60.2
直行或左转箭头	85.3	71.9	65.8	66.3
圆形井盖	81.4	66.9	60.7	58.7
方形井盖	84.3	71.3	70.1	58.6
黄色车道线	88.3	72.2	68.5	64.4
平均交并比	86.1	62.7	58.1	57.6