J Shanghai Jiaotong Univ Sci ›› 2021, Vol. 26 ›› Issue (5): 561-568.doi: 10.1007/s12204-021-2345-x

• Intelligent Connected Vehicle • Previous Articles     Next Articles

Camera-Radar Fusion Sensing System Based on Multi-Layer Perceptron

YAO Tonga (姚 彤), WANG Chunxianga (王春香), QIAN Yeqiangb (钱烨强)   

  1. (a. Department of Automation; b. University of Michigan - Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University, Shanghai 200240, China)
  • Received:2021-02-05 Online:2021-10-28 Published:2021-10-28

Abstract: Environmental perception is a key technology for autonomous driving. Owing to the limitations of a single sensor, multiple sensors are often used in practical applications. However, multi-sensor fusion faces some problems, such as the choice of sensors and fusion methods. To solve these issues, we proposed a machine learning-based fusion sensing system that uses a camera and radar, and that can be used in intelligent vehicles. First, the object detection algorithm is used to detect the image obtained by the camera; in sequence, the radar data is preprocessed, coordinate transformation is performed, and a multi-layer perceptron model for correlating the camera detection results with the radar data is proposed. The proposed fusion sensing system was verified by comparative experiments in a real-world environment. The experimental results show that the system can effectively integrate camera and radar data results, and obtain accurate and comprehensive object information in front of intelligent vehicles.

Key words: intelligent vehicle, environmental perception system, sensor fusion, multi-layer perceptron

CLC Number: