Journal of Shanghai Jiao Tong University ›› 2021, Vol. 55 ›› Issue (5): 607-614.doi: 10.16183/j.cnki.jsjtu.2020.120

Special Issue: 《上海交通大学学报》2021年12期专题汇总专辑 《上海交通大学学报》2021年“自动化技术、计算机技术”专题

Previous Articles     Next Articles

Video Abnormal Detection Combining FCN with LSTM

WU Guangli1,2(), GUO Zhenzhou1, LI Leiting1, WANG Chengxiang1   

  1. 1. School of Cyber Security, Gansu University of Political Science and Law, Lanzhou 730070, China
    2. Key Laboratory of China’s Ethnic Languages and Information Technology of the Ministry of Education, Northwest Minzu University, Lanzhou 730030, China;
  • Received:2020-04-26 Online:2021-05-28 Published:2021-06-01


In view of the shortcomings of the traditional video anomaly detection model, a network structure combining the fully convolutional neural (FCN) network and the long short-term memory (LSTM)network is proposed. The network can perform pixel-level prediction and can accurately locate abnormal areas. The network first uses the convolutional neural network to extract image features of different depths in video frames. Then, different image features are input to memory network to analyze semantic information on time series. Image features and semantic information are fused through residual structure. At the same time, the skip structure is used to integrate the fusion features in multi-mode and upsampling is conducted to obtain a prediction image with the same size as the original video frame. The proposed model is tested on the ped 2 subset of University of California, San Diego (UCSD) anomaly detection dataset and University of Minnesota System(UMN)crowd activity dataset. And both two datasets achieve good results. On the UCSD dataset, the equal error rate is as low as 6.6%, the area under curve reaches 98.2%, and the F1 score reaches 94.96%. On the UMN dataset, the equal error rate is as low as 7.1%, the area under curve reaches 93.7%, and the F1 score reaches 94.46%.

Key words: computer vision, video abnormal detection, pixel-level prediction, full convolutional neural (FCN) network, long short-term memory (LSTM) network

CLC Number: