To achieve precise localization, autonomous vehicles usually rely on a multi-sensor perception system
surrounding the mobile platform. Calibration is a time-consuming process, and mechanical distortion will cause
extrinsic calibration errors. Therefore, we propose a lidar-visual-inertial odometry, which is combined with an
adapted sliding window mechanism and allows for online nonlinear optimization and extrinsic calibration. In the
adapted sliding window mechanism, spatial-temporal alignment is performed to manage measurements arriving at
different frequencies. In nonlinear optimization with online calibration, visual features, cloud features, and inertial
measurement unit (IMU) measurements are used to estimate the ego-motion and perform extrinsic calibration.
Extensive experiments were carried out on both public datasets and real-world scenarios. Results indicate that the
proposed system outperforms state-of-the-art open-source methods when facing challenging sensor-degenerating
conditions.
MAO Tianyang (茅天阳), ZHAO Wentao (赵文韬),
WANG Jingchuan∗ (王景川), CHEN Weidong (陈卫东)
. Lidar-Visual-Inertial Odometry with Online Extrinsic Calibration[J]. Journal of Shanghai Jiaotong University(Science), 2023
, 28(1)
: 70
-76
.
DOI: 10.1007/s12204-023-2570-6
[1] ZHUANG H Y, ZHOU X J, WANG C X, et al. Wavelet transform-based high-definition map construction from a panoramic camera [J]. Journal of Shanghai Jiao Tong University (Science), 2021, 26(5): 569-576.
[2] CHEN J C, LI L, YANG X B. Efficient online vehicle tracking for real–virtual mapping systems [J]. Journal of Shanghai Jiao Tong University (Science), 2021, 26(5): 598-606.
[3] AN L F, ZHANG X Y, GAO H B, et al. Semantic segmentation–aided visual odometry for urban autonomous driving [J]. International Journal of Advanced Robotic Systems, 2017, 14(5): 1-11.
[4] MOURIKIS A I, ROUMELIOTIS S I. A multi-state constraint Kalman filter for vision-aided inertial navigation [C]//2007 IEEE International Conference on Robotics and Automation. Rome: IEEE, 2007: 3565-3572.
[5] LI M Y, MOURIKIS A I. High-precision, consistent EKF-based visual-inertial odometry [J]. The International Journal of Robotics Research, 2013, 32(6): 690-711.
[6] QIN T, LI P L, SHEN S J. VINS-Mono: A robust and versatile monocular visual-inertial state estimator [J]. IEEE Transactions on Robotics, 2018, 34(4): 1004-1020.
[7] ZHANG J, SINGH S. Low-drift and real-time lidar odometry and mapping [J]. Autonomous Robots, 2017, 41(2): 401-416.
[8] SHAN T X, ENGLOT B, MEYERS D, et al. LIOSAM: Tightly-coupled lidar inertial odometry via smoothing and mapping [C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas, NV: IEEE, 2020: 5135-5142.
[9] XU W, ZHANG F. FAST-LIO: A fast, robust LiDAR-inertial odometry package by tightly-coupled iteratedKalman filter [J]. IEEE Robotics and Automation Letters, 2021, 6(2): 3317-3324.
[10] PEREIRA M, et al. Self calibration of multiple LIDARs and cameras on autonomous vehicles [J]. Robotics and Autonomous Systems, 2016, 83: 326-337.
[11] KIM D H, KIM G W. Efficient calibration method of multiple LiDARs on autonomous vehicle platform [C]//2020 IEEE International Conference on Big Data and Smart Computing. Busan: IEEE, 2020: 446-447.
[12] GOGINENI S. Multi-sensor fusion and sensor calibration for autonomous vehicles [J]. International Research Journal of Engineering and Technology, 2020, 7(7): 1073-1078.
[13] BELTR′AN J, GUINDEL C, DE LA ESCALERA A, et al. Automatic extrinsic calibration method for LiDAR and camera sensor setups [J]. Transactions on Intelligent Transportation Systems, 2022, 23(10): 1-13.
[14] KATO S, TAKEUCHI E, ISHIGURO Y, et al. An open approach to autonomous vehicles [J]. IEEE Micro, 2015, 35(6): 60-68.
[15] FURGALE P, BARFOOT T D, SIBLEY G. Continuous-time batch estimation using temporal basis functions [C]//2012 IEEE International Conference on Robotics and Automation. Saint Paul, MN: IEEE,
2012: 2088-2095.
[16] LEVINSON J, THRUN S. Automatic online calibration of cameras and lasers [C]//Robotics: Science and Systems IX. Berlin: IEEE, 2013: 24-28.
[17] JIAO J H, YE H Y, ZHU Y L, et al. Robust odometry and mapping for multi-LiDAR systems with online extrinsic calibration [J]. IEEE Transactions on Robotics, 2022, 38(1): 351-371.
[18] ZUO X X, GENEVA P, LEE W, et al. LIC-fusion: LiDAR-inertial-camera odometry [C]//2019 IEEE/RSJ International Conference on Intelligent Robots and Systems. Macao: IEEE, 2019: 5848-5854.
[19] LIN J R, ZHENG C R, XU W, et al. R2LIVE: A robust, real-time, LiDAR-inertial-visual tightly-coupled state estimator and mapping [J]. IEEE Robotics and Automation Letters, 2021, 6(4): 7469-7476.
[20] SHAN T X, ENGLOT B, RATTI C, et al. LVI-SAM: Tightly-coupled lidar-visual-inertial odometry via smoothing and mapping [C]//2021 IEEE International Conference on Robotics and Automation. Xi’an: IEEE, 2021: 5692-5698.
[21] LUCAS B D, KANADE T. An iterative image registration technique with an application to stereo vision [C]//7th International Joint Conference on Artificial Intelligence. Vancouver: Morgan Kaufmann Publishers Inc, 1981: 674-679.
[22] HUBER P J. Robust estimation of a location parameter [M]//Breakthroughs in statistics. New York: Springer, 1992: 492-518.
[23] AGARWAL S, MIERLE K. Ceres solver: Tutorial & reference [EB/OL]. [2021-12-10]. http://www.ceressolver.org.
