Infrared and Visible Image Fusion Based on Region of Interest Detection and Nonsubsampled Contourlet Transform

doi:10.1007/s12204-013-1437-7

Journal of shanghai Jiaotong University (Science) ›› 2013, Vol. 18 ›› Issue (5): 526-534.doi: 10.1007/s12204-013-1437-7

Previous Articles Next Articles

Infrared and Visible Image Fusion Based on Region of Interest Detection and Nonsubsampled Contourlet Transform

LIU Huan-xi1,2* (刘欢喜), ZHU Tian-hong1 (朱天竑), ZHAO Jia-jia3 (赵佳佳)

(1. School of Electronic, Information and Electrical Engineering, Shanghai Jiaotong University, Shanghai 200240, China; 2. Jiangsu Junlong Power Technology Co., Ltd., Jingjiang 214500, Jiangsu, China; 3. Beijing Institute of Mechanical and Electrical Engineering, Beijing 100074, China)
(1. School of Electronic, Information and Electrical Engineering, Shanghai Jiaotong University, Shanghai 200240, China; 2. Jiangsu Junlong Power Technology Co., Ltd., Jingjiang 214500, Jiangsu, China; 3. Beijing Institute of Mechanical and Electrical Engineering, Beijing 100074, China)

Online:2013-10-31 Published:2013-12-05
Contact: LIU Huan-xi(刘欢喜) E-mail:lhxsjtu@sjtu.edu.cn

Abstract

Abstract: In order to enhance the contrast of the fused image and reduce the loss of fine details in the process of image fusion, a novel fusion algorithm of infrared and visible images is proposed. First of all, regions of interest (RoIs) are detected in two original images by using saliency map. Then, nonsubsampled contourlet transform (NSCT) on both the infrared image and the visible image is performed to get a low-frequency sub-band and a certain amount of high-frequency sub-bands. Subsequently, the coefficients of all sub-bands are classified into four categories based on the result of RoI detection: the region of interest in the low-frequency sub-band (LSRoI), the region of interest in the high-frequency sub-band (HSRoI), the region of non-interest in the low-frequency sub-band (LSNRoI) and the region of non-interest in the high-frequency sub-band (HSNRoI). Fusion rules are customized for each kind of coefficients and fused image is achieved by performing the inverse NSCT to the fused coefficients. Experimental results show that the fusion scheme proposed in this paper achieves better effect than the other fusion algorithms both in visual effect and quantitative metrics.

Key words: image fusion| region of interest (RoI) detection| saliency map| nonsubsampled contourlet transform (NSCT)

摘要： In order to enhance the contrast of the fused image and reduce the loss of fine details in the process of image fusion, a novel fusion algorithm of infrared and visible images is proposed. First of all, regions of interest (RoIs) are detected in two original images by using saliency map. Then, nonsubsampled contourlet transform (NSCT) on both the infrared image and the visible image is performed to get a low-frequency sub-band and a certain amount of high-frequency sub-bands. Subsequently, the coefficients of all sub-bands are classified into four categories based on the result of RoI detection: the region of interest in the low-frequency sub-band (LSRoI), the region of interest in the high-frequency sub-band (HSRoI), the region of non-interest in the low-frequency sub-band (LSNRoI) and the region of non-interest in the high-frequency sub-band (HSNRoI). Fusion rules are customized for each kind of coefficients and fused image is achieved by performing the inverse NSCT to the fused coefficients. Experimental results show that the fusion scheme proposed in this paper achieves better effect than the other fusion algorithms both in visual effect and quantitative metrics.

关键词: image fusion| region of interest (RoI) detection| saliency map| nonsubsampled contourlet transform (NSCT)

CLC Number:

TP 391.4

LIU Huan-xi1,2* (刘欢喜), ZHU Tian-hong1 (朱天竑), ZHAO Jia-jia3 (赵佳佳). Infrared and Visible Image Fusion Based on Region of Interest Detection and Nonsubsampled Contourlet Transform[J]. Journal of shanghai Jiaotong University (Science), 2013, 18(5): 526-534.

References

[1] Stathaki T. Image fusion: Algorithms and applications [M]. New York: Academic Press, 2008.
[2] Rockinger O, Fechner T. Pixel-level image fusion: The case of image sequences [J]. Proceedings of SPIE, 1998, 3374: 378-388.
[3] Burt P J, Adelson E H. Merging images through pattern decomposition [J]. Proceedings of SPIE, 1985, 575: 173-181.
[4] Toet A, Van Ruyven L J, Valeton J M. Merging thermal and visual images by a contrast pyramid [J]. Optical Engineering, 1989, 28(7): 789-792.
[5] Burt P J, Kolczynski R J. Enhanced image capture through fusion [C]//Proceedings of the 4th International Conference on Computer Vision. Berlin: IEEE, 1993: 173-182.
[6] Hill P R, Bull D R, Canagarajah C N. Image fusion using a new framework for complex wavelet transforms [C]//Proceedings of the IEEE International Conference on Image Processing. Genoa: IEEE, 2005: 1338-1341.
[7] Li S T, Wang Y N. Landsat TM and SAR image fusion by multi-wavelet transform [J]. Proceedings of SPIE. 2001, 4548: 146-150.
[8] Simoncelli E P, Freeman W T. The steerable pyramid: A flexible architecture for multi-scale derivative computation [C]//Proceedings of International Conference on Image Processing. Washington DC: IEEE, 1995: 444-447.
[9] Candes E J, Donoho D L. Curvelets: A suprizingly effective nonadaptive representation for objects with edges [C]//Saint-Malo Proceedings. Nashville, TN: Vanderbilt University Press, 1999: 1-16.
[10] Do M N, Vetterli M. Beyond wavelets: Contourlets [M]. New York: Academic Press, 2003.
[11] Cunha A L, Zhou J P, Donoho M N. The nonsubsampled contourlet transform: Theory, design, and applications [J]. IEEE Transactions on Image Processing, 2006, 15(10): 3089-3101.
[12] Hou X D, Zhang L Q. Saliency detection: A spectral residual approach [C]// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN: IEEE, 2007: 1-8.
[13] Lewis J J, Nikolov S G, Canagarajah C N, et al. Uni-modal versus joint segmentation for regionbased image fusion [C]//Proceedings of the 9th International Conference on Information Fusion. Florence, Italy: IEEE, 2006: 10-13.
[14] Doyle W. Operation useful for similarity-invariant pattern recognition [J]. Journal of Association of Computer Mechanics, 1962, 9(2): 259-267.

Infrared and Visible Image Fusion Based on Region of Interest Detection and Nonsubsampled Contourlet Transform

Infrared and Visible Image Fusion Based on Region of Interest Detection and Nonsubsampled Contourlet Transform

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 9

Recommended Articles

Metrics

Comments

[1]	WANG Yili, LI Qiang, SHEN Junyi, YANG Yidong, WANG Qi. Multi-Modal Warship Image Generation and Its Key Part Detection [J]. Air & Space Defense, 2025, 8(1): 77-85.
[2]	CHANG Qing∗ (常青), YANG Wenyou (杨文友), CHEN Lanlan (陈兰岚). Deformable Registration Algorithm via Non-subsampled Contourlet Transform and Saliency Map [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(4): 452-462.
[3]	WANG Rui,DU Linfeng,CHEN Junli,WAN Wanggen. Image Fusion Based on Random Projection and Sparse Representation [J]. Journal of Shanghai Jiaotong University, 2014, 48(10): 1421-1427.
[4]	HAO Min-Hui, LIU Zhe, ZHANG Yong-Liang, ZHANG He-Ni. An Improved Total Variation-Based Image Fusion Algorithm [J]. Journal of Shanghai Jiaotong University, 2013, 47(04): 607-612.
[5]	SONG Hao-hao (宋好好), LU Zhen (陆臻). Image Fusion Scheme Based on Nonsubsampled Contourlet and Block-Based Cosine Transform [J]. J Shanghai Jiaotong Univ Sci, 2012, 17(1): 8-012.
[6]	WU Yi-quan (吴一全), WU Chao (吴超), WU Shi-hua (吴诗婳) . Fusion of Remote Sensing Images Based on Nonsubsampled Contourlet Transform and Region Segmentation [J]. Journal of shanghai Jiaotong University (Science), 2011, 16(6): 722-727.
[7]	CHEN Shuo, YU Xiao-Sheng, WU Cheng-Dong, CHEN Dong-Yue. Salient Building Region Detection for Scene Classification [J]. Journal of Shanghai Jiaotong University, 2011, 45(08): 1130-1135.
[8]	CAO Yin,CAI Yunze,HAN Yu . A New Fusion Method for Lostfocus Images Based on Vision Structural Similarity [J]. Journal of Shanghai Jiaotong University, 2010, 44(11): 1507-1512.
[9]	XU Xian，YANG Jie. High Degree of Accuracy Medical Image Fusion Algorithm Based on GPU Hardware Acceleration [J]. Journal of Shanghai Jiaotong University, 2010, 44(02): 248-0251.