Infrared Single Pixel Imaging Based on Generative Adversarial Network

doi:10.1007/s12204-023-2654-3

Abstract

Abstract: In the field of imaging, the image resolution is required to be higher. There is always a contradiction between the sensitivity and resolution of the seeker in the infrared guidance system. This work uses the rosette scanning mode for physical compression imaging in order to improve the resolution of the image as much as possible under the high-sensitivity infrared rosette point scanning mode and complete the missing information that is not scanned. It is effective to use optical lens instead of traditional optical reflection system, which can reduce the loss in optical path transmission. At the same time, deep learning neural network is used for control. An infrared single pixel imaging system that integrates sparse algorithm and recovery algorithm through the improved generative adversarial networks is trained. The experiment on the infrared aerial target dataset shows that when the input is sparse image after rose sampling, the system finally can realize the single pixel recovery imaging of the infrared image, which improves the resolution of the image while ensuring high sensitivity.

Key words: image resolution, rose sampling, generative adversarial networks, single pixel imaging

摘要： 成像领域中对图像的分辨率要求越来越高，对于红外制导系统，始终存在导引头灵敏度与分辨率之间的矛盾。为解决在高灵敏度的红外玫瑰线点扫描方式下尽可能地提高图像分辨率，补全未扫描到的缺失信息，本项工作使用玫瑰线扫描的方式进行物理方式压缩成像。使用光学透镜替代传统的光学反射系统可以有效减少光路传输中损耗，同时结合深度学习神经网络进行控制，通过改进的生成式对抗网络，训练出一个集稀疏算法与恢复算法为一体的红外单像素成像系统。在红外空中目标数据集上的实验表明：输入为玫瑰线采样后的稀疏图像时，最终实现红外图像的单像素恢复成像，在保证高灵敏度的同时提高了图像分辨率。

关键词: 图像分辨率，玫瑰线扫描，生成式对抗网络，单像素成像

CLC Number:

TP391

JIANG Yilin, ZHANG Yilong, ZHANG Fangyuan. Infrared Single Pixel Imaging Based on Generative Adversarial Network[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(6): 1114-1124.

References

[1] ZHANG Z J, LIU L, LI X R, et al. Compressed sensing for rapid IR imaging [C]//IET Colloquium on Millimetre-Wave and Terahertz Engineering & Technology 2016. London: IET, 2016: 1-6.

[2] UZELER H, CAKIR S, AYTAÇ T. Image reconstruction for single detector rosette scanning systems based on compressive sensing theory [J]. Optical Engineering, 2016, 55(2): 023108.

[3] XIE C, LU X, ZENG W. Single frame super-resolution reconstruction based on sparse representation [J]. Journal of Southeast University (English Edition), 2016, 32(2): 177-182.

[4] BROMBERG Y, KATZ O, SILBERBERG Y. Ghost imaging with a single detector [J]. Physical Review A, 2009, 79(5): 053840.

[5] SHAPIRO J H. Computational ghost imaging [J]. Physical Review A, 2008, 78(6): 061802.

[6] WANG L, ZHAO S M. Fast reconstructed and high-quality ghost imaging with fast Walsh–Hadamard transform [J]. Photonics Research, 2016, 4(6): 240.

[7] ZHANG Z B, LIU S J, PENG J Z, et al. Simultaneous spatial, spectral, and 3D compressive imaging via efficient Fourier single-pixel measurements [J]. Optica, 2018, 5(3): 315.

[8] ROUSSET F, DUCROS N, FARINA A, et al. Adaptive basis scan by wavelet prediction for single-pixel imaging [J]. IEEE Transactions on Computational Imaging, 2017, 3(1): 36-46.

[9] TSAI R, HUANG T S. Multiframe image restoration and registration [J]. Computer Vision and Image Processing, 1984, 1: 317-339.

[10] KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1637-1645.

[11] ZHANG D, HE J Z. Hybrid sparse-representation-based approach to image super-resolution reconstruction [J]. Journal of Electronic Imaging, 2017, 26(2): 023008.

[12] TAN J, TAO Z Q, CAO A H, et al. An edge-preserving iterative back-projection method for image super-resolution [J]. Proceedings of SPIE, 2016, 10033: 844-849.

[13] DAVENPORT M A, WAKIN M B. Analysis of orthogonal matching pursuit using the restricted isometry property [J]. IEEE Transactions on Information Theory, 2010, 56(9): 4395-4401.

[14] TROPP J A, GILBERT A C. Signal recovery from random measurements via orthogonal matching pursuit [J]. IEEE Transactions on Information Theory, 2007, 53(12): 4655-4666.

[15] YANG J C, WRIGHT J, HUANG T S, et al. Image super-resolution via sparse representation [J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.

[16] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE, 2017: 1132-1140.

[17] ZHANG Y L, TIAN Y P, KONG Y, et al. Residual dense network for image super-resolution [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2472-2481.

[18] AN Z Y, ZHANG J Y, SHENG Z Y, et al. RBDN: Residual bottleneck dense network for image super-resolution [J]. IEEE Access, 2021, 9: 103440-103451.

[19] ZHU Y, GEIß C, SO E. Image super-resolution with dense-sampling residual channel-spatial attention networks for multi-temporal remote sensing image classification [J]. International Journal of Applied Earth Observation and Geoinformation, 2021, 104: 102543.

[20] ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks [M]//Computer vision - ECCV 2018. Cham: Springer, 2018: 294-310.

[21] WANG Z H, CHEN J, HOI S C H. Deep learning for image super-resolution: A survey [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3365-3387.

[22] AYAS S, EKINCI M. Microscopic image super resolution using deep convolutional neural networks [J]. Multimedia Tools and Applications, 2020, 79(21): 15397-15415.

[23] WANG Y F, PERAZZI F, MCWILLIAMS B, et al. A fully progressive approach to single-image super-resolution [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE, 2018: 977-97709.

[24] SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1874-1883.

[25] CABALLERO J, LEDIG C, AITKEN A, et al. Real-time video super-resolution with spatio-temporal networks and motion compensation [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2848-2857.

[26] SAJJADI M S M, SCHÖLKOPF B, HIRSCH M. EnhanceNet: single image super-resolution through automated texture synthesis [C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4501-4510.

[27] WANG X T, YU K, DONG C, et al. Recovering realistic texture in image super-resolution by deep spatial feature transform [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 606-615.