Real-Time Ranging of Vehicles and Pedestrians for Mobile Application on Smartphones

doi:10.1007/s12204-022-2411-z

Abstract

Abstract: The vehicles and pedestrians ranging is one of the basic functions of advanced driving assistance system. However, most of the ranging systems can only work on workstations with high computing power. To solve this problem, a lightweight algorithm is proposed to be packaged into Android application package, and be installed in Android smartphones for vehicles and pedestrians ranging. The proposed ranging system is based on the images obtained by smartphone’s monocular camera. To achieve real-time ranging, an 8-bit integer (int8) quantization algorithm is proposed to accelerate the inference of convolutional neural networks. To increase the detection precision, a zoom-in algorithm is further proposed to detect small targets in the distance. After having detected the 2D bounding boxes of vehicles and pedestrians, a pinhole ranging method is applied to estimate the distance. In order to verify the proposed algorithm, the mean average precision (mAP) and the frame per second (FPS) are first tested by using COCO dataset on Huawei P40Pro, then, the ranging precision on the real road.The experimental results show that this algorithm can successfully perform real-time ranging (15 FPS) with high precision (34.8 mAP) onto the tested smartphones. Finally, a possible mobile application based on the ranging algorithm, i.e., distance keeping warning, is also provided.

Key words: advanced driving assistance system, computer vision, int8 quantization, visual ranging, smartphone application

摘要： 车辆与行人测距是高级驾驶辅助系统的基本功能之一。然而，大多数测距系统只能在具有高计算能力的工作站上工作。为了解决这一问题，提出了一种轻量级算法，将其打包到Android应用程序包中，安装在Android智能手机中，用于车辆和行人测距。该测距系统基于智能手机单目摄像头获取的图像。为了实现实时测距，提出了一种8位整数（int8）量化算法来加速卷积神经网络的推理。为了提高检测精度，进一步提出了一种放大算法来检测远距离的小目标。在检测到车辆和行人的二维边界框后，采用针孔测距法估计距离。为了验证所提出的算法，首先在华为P40Pro上使用COCO数据集测试了平均精度均值（mAP）和帧/秒（FPS），然后在真实道路上测试了测距精度。实验结果表明：该算法能够在被测智能手机上成功实现34.8 mAP高精度的15 FPS实时测距。最后，给出了基于测距算法的一种可能的移动应用，即保持距离预警。

关键词: 高级驾驶辅助系统，计算机视觉，int8量化，视觉测距，智能手机应用

CLC Number:

TP391.4
U463

ZHOU Su (周苏), ZHONG Zebin^∗ (钟泽滨). Real-Time Ranging of Vehicles and Pedestrians for Mobile Application on Smartphones[J]. J Shanghai Jiaotong Univ Sci, 2024, 29(6): 1081-1090.

References

[1] YU X, MARINOV M. A study on recent developments and issues with obstacle detection systems for automated vehicles [J]. Sustainability, 2020, 12(8): 3281.
[2] WANG Z, LIU J. A review of object detection based on convolutional neural network [C]//2017 36th Chinese Control Conference. Dalian: IEEE, 2017: 11101-11109.
[3] CHEY Y H. Vehicle detection and collision avoidance apparatus: US4626850 [P]. 1986-12-02.
[4] VIOLA P, JONES M J, SNOW D. Detecting pedestrians using patterns of motion and appearance [J]. International Journal of Computer Vision, 2005, 63(2):153-161.
[5] MALISIEWICZ T, GUPTA A, EFROS A A. Ensemble of exemplar-SVMs for object detection and beyond [C]//2011 International Conference on Computer Vision. Barcelona, Spain: IEEE, 2011: 89-96.
[6] KRIZHEVSKY A, SUTSKEVER I, HINTON G E.ImageNet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017,60(6): 84-90.
[7] BOCHKOVSKIY A, WANG C Y, LIAO H Y M.YOLOv4: Optimal speed and accuracy of object detection [EB/OL]. (2020-04-23). https://arxiv.org/abs/2004.10934.
[8] POTMESIL M, CHAKRAVARTY I. Synthetic image generation with a lens and aperture camera model [J].ACM Transactions on Graphics, 1982, 1(2): 85-108.
[9] NAKAMURA K, ISHIGAKI K, OGATA T, et al.Real-time monocular ranging by Bayesian triangulation [C]//2013 IEEE Intelligent Vehicles Symposium.Gold Coast: IEEE, 2013: 1368-1373.
[10] GAO M, MENG X, YANG Y, et al. A traffic avoidance system based on binocular ranging [C]//2017 12th IEEE Conference on Industrial Electronics and Applications. Siem Reap: IEEE, 2017: 1039-1043.
[11] WU J, LENG C, WANG Y, et al. Quantized convolutional neural networks for mobile devices [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016: 4820-4828.
[12] HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetv3 [C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019:1314-1324.
[13] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection[C]//2016 IEEE Conference on Computer Vision andPattern Recognition. Las Vegas, NV: IEEE, 2016: 779-788.
[14] JOCHER G, STOKEN A, BOROVEC J, et al. Ultralytics/yolov5: v3.1 - Bug fixes and performance improvements [EB/OL]. (2020-10-29). https://doi.org/10.5281/zenodo.4154370.
[15] JACOB B, KLIGYS S, CHEN B, et al. Quantization and training of neural networks for efficient integerarithmetic-only inference [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City, UT: IEEE, 2018: 2704-2713.
[16] KRISHNAMOORTHI R. Quantizing deep convolutional networks for efficient inference: A whitepaper [EB/OL]. (2018-06-21). https://arxiv.org/abs/1806.08342.
[17] GOOGLE. Tensorflow-Lite [EB/OL]. [2021-01-25].https://tensorflow.google.cn/lite.
[18] KULLBACK S, LEIBLER R A. On information and sufficiency [J]. The Annals of Mathematical Statistics,1951, 22(1): 79-86.
[19] SALIMANS T, KINGMA D P. Weight normalization:A simple reparameterization to accelerate training of deep neural networks [M]//Advances in neural information processing systems 29. Red Hook, NY: Curran Associates, Inc., 2016: 901-909.
[20] FUKUSHIMA K. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position [J]. Biological Cybernetics, 1980, 36(4): 193-202.
[21] ANYSZ H, ZBICIAK A, IBADOV N. The influence of input data standardization method on prediction accuracy of artificial neural networks [J]. Procedia Engineering, 2016, 153: 66-70.
[22] ROSER M, APPEL C, RITCHIE H. Human height [EB/OL]. [2021-01-25]. https:// ourworldindata.org/human-height.
[23] TOHA M T H. Average car length: List of car lengths in details [EB/OL]. [2021-01-25]. https:// anewwayforward.org/ average-car-length.
[24] GOOGLE. Android Studio 4.1.1 [EB/OL]. [2021-01-25]. https://developer.android.com/studio.
[25] LIN T Y, MAIRE M, BELONGIE SJ, et al. Microsoft COCO: Common objects in context [M]//Computer vision—ECCV 2014. Cham: Springer, 2014: 740-755.
[26] AARP. What is a safe following distance on the road[EB/OL]. [2021-01-25]. https:// www.aarp.org/ auto/driver-safety/ info-2013/ safe-following-distance.