J Shanghai Jiaotong Univ Sci ›› 2021, Vol. 26 ›› Issue (5): 587-597.doi: 10.1007/s12204-021-2348-7

• Intelligent Connected Vehicle • Previous Articles     Next Articles

Intelligent Analysis of Abnormal Vehicle Behavior Based on a Digital Twin

LI Lin1 (李 霖), HU Zeyu2 (胡泽宇), YANG Xubo2 (杨旭波)   

  1. (1. Shanghai International Automobile City (Group) Co., Ltd., Shanghai 201805, China; 2. School of Software, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China)
  • Received:2020-11-30 Online:2021-10-28 Published:2021-10-28

Abstract: Analyzing a vehicle’s abnormal behavior in surveillance videos is a challenging field, mainly due to the wide variety of anomaly cases and the complexity of surveillance videos. In this study, a novel intelligent vehicle behavior analysis framework based on a digital twin is proposed. First, detecting vehicles based on deep learning is implemented, and Kalman filtering and feature matching are used to track vehicles. Subsequently, the tracked vehicle is mapped to a digital-twin virtual scene developed in the Unity game engine, and each vehicle’s behavior is tested according to the customized detection conditions set up in the scene. The stored behavior data can be used to reconstruct the scene again in Unity for a secondary analysis. The experimental results using real videos from traffic cameras illustrate that the detection rate of the proposed framework is close to that of the state-of-the-art abnormal event detection systems. In addition, the implementation and analysis process show the usability, generalization, and effectiveness of the proposed framework.

Key words: digital twin, deep learning, vehicle detection, abnormal behavior

CLC Number: