FPGA Design of Image Defogging System in Intelligent Tachograph

doi:10.16183/j.cnki.jsjtu.2023.026

Abstract

Abstract:

In hazy weather, due to the low visibility of traffic roads, the video images collected are degraded and the image information is fuzzy. Considering the problem of low real-time processing of the traditional system, an image defogging system based on the ZYNQ platform is designed and applied to the intelligent driving recorder system. First, aiming at the problems of the traditional dark channel defogging algorithm in the sky region, a sky region segmentation strategy is proposed to correct the image restoration parameters. Then, in order to solve the problem that the pixel ordering of the whole image consumes a lot of resources when calculating the global atmospheric light value, a frame iteration method is proposed to optimize the atmospheric light value by making the advantage of the parallel operation on the FPGA platform, and at the same time, the optimization of guided filtering is realized. Finally, using dual-channel high definition multimedia interface (HDMI) resources, a real-time traffic image video processing experimental platform is established by using one channel of the dual HDMI resources as video input and the other as video processing output, and experimental simulation of the algorithm in this paper is conducted. The experimental results show that the system has a good defogging performance for the traffic video in hazy weather, especially in solving the problem of defogging distortion in the sky area. When the traffic video with a resolution of 1 280×720 is defogged, the processing speed can reach 30 frame/s meeting the real-time requirements.

Key words: traffic video, image defogging, ZYNQ platform, real-time processing

CLC Number:

TP391.41

HUANG He, HU Kaiyi, YANG Lan, WANG Hao, GAO Tao, WANG Huifeng. FPGA Design of Image Defogging System in Intelligent Tachograph[J]. Journal of Shanghai Jiao Tong University, 2024, 58(4): 565-578.

Figures/Tables 26

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Tab.1

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

Tab.2

Fig.20

Fig.21

Fig.22

Fig.23

Tab.3

References 18

[1]	苗开超, 姚叶青, 王传辉, 等. 一次导致重大交通事故的大雾天气成因分析[J]. 气象科技进展, 2021, 11(4): 23-27.
	MIAO Kaichao, YAO Yeqing, WANG Chuanhui, et al. Analysis of a dense fog event caused a fatal traffic accident[J]. Advances in Meteorological Science and Technology, 2021, 11(4): 23-27.
[2]	王洁, 王顺吉, 侯刚. 基于Zynq的嵌入式数字图像处理系统设计[J]. 实验室科学, 2020, 23(4): 70-73.
	WANG Jie, WANG Shunji, HOU Gang. Design of embedded image system based on Z7 series[J]. Laboratory Science, 2020, 23(4): 70-73.
[3]	黄鹤, 胡凯益, 郭璐, 等. 改进的海雾图像去除方法[J]. 哈尔滨工业大学学报, 2021, 53(8): 81-91.
	HUANG He, HU Kaiyi, GUO Lu, et al. Improved defogging algorithm for sea fog[J]. Journal of Harbin Institute of Technology, 2021, 53(8): 81-91.
[4]	CAI B L, XU X M, JIA K, et al. DehazeNet: An end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198. doi: 10.1109/TIP.2016.2598681 pmid: 28873058
[5]	黄鹤, 胡凯益, 宋京, 等. 雾霾线求解透射率的二次优化方法[J]. 西安交通大学学报, 2021, 55(8): 130-138.
	HUANG He, HU Kaiyi, SONG Jing, et al. A twice optimization method for solving transmittance with haze-lines[J]. Journal of Xi’an Jiaotong University, 2021, 55(8): 130-138.
[6]	NAYAR S K, NARASIMHAN S G. Vision in bad weather[C]// Proceedings of the Seventh IEEE International Conference on Computer Vision. New York, NY, USA: IEEE, 2002:598-605.
[7]	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353. doi: 10.1109/TPAMI.2010.168 pmid: 20820075
[8]	黄鹤, 李昕芮, 宋京, 等. 多尺度窗口的自适应透射率修复交通图像去雾方法[J]. 中国光学, 2019, 12(6): 1311-1320.
	HUANG He, LI Xinrui, SONG Jing, et al. A traffic image dehaze method based on adaptive transmittance estimation with multi-scale window[J]. Chinese Optics, 2019, 12(6): 1311-1320. doi: 10.3788/co. URL
[9]	尹震宇, 徐光远, 张飞青, 等. 面向Zynq平台的卷积神经网络单元设计与实现[J]. 小型微型计算机系统, 2022, 43(2): 231-235.
	YIN Zhenyu, XU Guangyuan, ZHANG Feiqing, et al. Design and implementation of convolution neural network unit based on Zynq platform[J]. Journal of Chinese Computer Systems, 2022, 43(2): 231-235.
[10]	KIM J H, KIM C S. Approximate solution to optimized contrast enhancement for hazy image[C]// IEEE International Symposium on Consumer Electronics. Seoul, Korea: IEEE, 2014: 1-2.
[11]	刘倩, 陈茂银, 周东华. 基于单幅图像的快速去雾算法[C]// 中国控制与决策会议. 贵阳: 控制与决策编辑部, 2013: 3781-3786.
	LIU Qian, CHEN Maoyin, ZHOU Donghua. Fast haze removal from a single image[C]// Chinese Control and Decision Conference. Guiyang, China: Control and Decision, 2013: 3781-3786.
[12]	刘光飞, 胡辽林. 暗通道先验去雾算法的改进及FPGA实现[J]. 西安理工大学学报, 2016, 32(1): 77-82.
	LIU Guangfei, HU Liaolin. Improvement and FPGA implementation of dark channel priori dehazing algorithm[J]. Journal of Xi’an University of Technology, 2016, 32(1): 77-82.
[13]	龚亮, 孙俊喜, 顾播宇, 等. 基于FPGA的视频图像去雾系统的设计与实现[J]. 电视技术, 2013, 37(7): 6-8.
	GONG Liang, SUN Junxi, GU Boyu, et al. Design and implementation of haze removal system of video image based on FPGA[J]. Video Engineering, 2013, 37(7): 6-8.
[14]	LIU H, HUANG D, HOU S, et al. Large size single image fast defogging and the real time video defogging FPGA architecture[J]. Neurocomputing, 2017, 269(20): 97-107. doi: 10.1016/j.neucom.2016.09.139 URL
[15]	BEGUM R, RAJAPRIYA S, KANNAN G. FPGA implementation and simulation of guided image filtering[C]// International Conference Icats. Thanjavur, India: ICATS, 2014:218-227.
[16]	黄文辉, 陈仁雷, 张家谋. 数字视频图像质量客观测量方法的改进与实现[J]. 北京邮电大学学报, 2005(4): 87-90. doi: 10.13190/jbupt.200504.87.huangwh
	HUANG Wenhui, CHEN Renlei, ZHANG Jiamou. Improvement and implementation of objective digital video quality measurement[J]. Journal of Beijing University of Posts and Telecommunications, 2005(4): 87-90.
[17]	黄鹤, 胡凯益, 李战一, 等. 融合MCAP和GRTV正则化的无人机航拍建筑物图像去雾方法[J]. 上海交通大学学报, 2023, 57(3): 366-378. doi: 10.16183/j.cnki.jsjtu.2021.238
	HUANG He, HU Kaiyi, LI Zhanyi, et al. An image dehazing method for UAV aerial photography of buildings combining MCAP and GRTV regularization[J]. Journal of Shanghai Jiao Tong University, 2023, 57(3): 366-378.
[18]	CHOI L K, YOU J, BOVIK A C. Referenceless prediction of perceptual fog density and perceptual image defogging[J]. IEEE Transactions on Image Processing, 2015, 24(11): 3888-3901. doi: 10.1109/TIP.2015.2456502 pmid: 26186784

子模块名称	功能说明	输入输出说明
天空区域检测模块	天空区域分割	输入帧缓存图像输出分割后的二值数据
暗通道模块	暗通道图像求取	输入RGB图像输出为暗通道像素数据
大气光值求取模块	大气光值计算	天空分割模块中间均值变量阈值倍数
透射率求取模块	图像透射率计算	输入暗通道图像及其全局大气光值,输出粗估计透射率
透射率优化模块	透射率修正	输入灰度图像及粗估计透射率,输出细化透射率
图像复原模块	去雾图像复原	输入RGB图像、细化后的透射率、大气光值,输出去雾图像数据

实验组	去雾算法	H	g_A	r_P	E_FAD	对比度
E1	原始图像	12.600 0	2.844 9	—	1.992 0	53.387 8
	He算法^[7]	14.663 9	4.685 0	24.213 1	0.884 3	62.995 2
	Kim算法^[10]	11.641 7	4.116 7	23.163 6	0.768 1	79.576 6
	刘算法^[11]	13.156 0	2.935 5	35.343 3	1.873 8	54.430 2
	本文算法	14.974 4	4.846 3	26.071 3	0.760 2	81.383 2
E2	原始图像	13.212 2	3.843 6	—	1.741 4	44.075 4
	He算法^[7]	14.781 8	6.420 3	24.200 1	0.749 9	51.237 1
	Kim算法^[10]	13.719 7	6.038 4	25.274 1	0.856 5	63.087 5
	刘算法^[11]	13.648 0	3.959 5	37.919 8	1.644 1	44.780 5
	本文算法	14.913 9	6.597 0	26.482 5	0.528 1	65.589 6
E3	原始图像	10.693 7	3.469 8	—	2.181 4	50.887 0
	He算法^[7]	11.863 2	5.262 3	24.084 5	0.744 7	45.300 1
	Kim算法^[10]	12.108 4	5.338 7	25.670 9	0.838 1	65.266 6
	刘算法^[11]	11.076 4	3.576 7	33.060 1	2.081 9	51.540 2
	本文算法	12.861 7	5.386 7	26.276 1	0.632 0	67.045 0
E4	原始图像	12.283 6	1.323 8	—	4.135 6	50.069 3
	He算法^[7]	14.207 5	3.277 0	17.995 6	0.966 2	50.141 4
	Kim算法^[10]	12.504 7	3.151 3	18.171 8	0.749 6	79.225 2
	刘算法^[11]	12.950 1	1.407 7	32.636 9	3.794 3	51.810 8
	本文算法	14.391 8	3.597 8	21.098 0	0.724 5	79.482 0

PL资源	可用资源	占用资源	占用率/%
LUT	56 200	19 143	34.1
FF	106 400	19 353	18.2
Block RAM	140	62	44.3
DSP	220	137	62.2