Hyperspectral Image Classification Method Based on Global Space-Spectral Attention Mechanism

doi:10.1007/s12204-024-2792-2

Abstract

Abstract: In hyperspectral remote sensing imagery, pixel interactions within defined spatial extents result in the mixing of adjacent pixels. Additionally, the high similarity of adjacent spectra leads to information redundancy,which hinders the extraction of global spatial and spectral correlations. In order to solve the problems of mixed adjacent pixels and redundant adjacent spectra, this work offers a hyperspectral image classification approach that uses a global space-spectral attention mechanism. First, the proposed method's global spatial attention module uses multi-scale dilated convolution to produce a bigger receptive field to be capable of capturing global spatial correlation and obtain unmixed pixel information. Then, the global spectral attention module designs a spectral domain partition algorithm, using the combination of regional density as well as information entropy as the threshold to divide spectrum into dispersed subsets and eliminate redundant information. The global context information for entire spectral band is fully exploited, and correlation of the global spectral information is extracted. Finally, the two modules combine to provide a global correlation of space and spectrum. Experiments demonstrate that the suggested method obtains overall accuracies of 97.28%, 94.73%, and 95.76% on the three WHU-Hi hyperspectral datasets, surpassing comparison methods.

Key words: remote sensing image, image classification, attention mechanism, hyperspectral image

摘要： 高光谱遥感图像在一定的空间范围内存在像元相互影响的问题，使得相邻像元混合。同时相邻光谱相似性高导致信息冗余，阻碍了全局空间和光谱相关性的提取。针对像元混合和光谱冗余的问题, 提出了一种基于全局空谱注意力的高光谱图像分类方法。首先, 全局空间注意力模块使用多尺度的扩张卷积获得更大的感受野，以捕捉全局空间的关联性，并获得解混的像元信息。然后，全局光谱注意力模块设计光谱域划分算法，以局部密度和信息熵的乘积为阈值，将光谱划分为分散的子集，消除冗余信息。通过充分挖掘整个谱带的全局上下文信息，提取全局光谱的关联性。最后，串联两个模块，得到全局的空间和光谱的关联性，以提高分类精度。实验表明：所提方法在WHU-Hi三个高光谱数据集上总体准确率分别达到了97.28%、94.73%和95.76%，超过了大部分现有方法。

关键词: 遥感图像，图像分类，注意力机制，高光谱图像

CLC Number:

TP391.41

Qin Rui, Wu Benze, Liu Xinfu, Wu Yirui. Hyperspectral Image Classification Method Based on Global Space-Spectral Attention Mechanism[J]. J Shanghai Jiaotong Univ Sci, 2026, 31(4): 909-919.

References

[1] ZHONG Z L, LI J, LUO Z M, et al. Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework [J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2): 847-858.
[2] WANG T, HU S M, SUN J G. Image retrieval based on color-spatial feature [J]. Journal of Software, 2002, 13(10): 2031-2036 (in Chinese).
[3] LI X M, ZHENG L, HU Z Y. SIFT based automatic registration of remotely-sensed imagery [J]. Journal of Remote Sensing, 2006, 10(6): 885-892 (in Chinese).
[4] QI S X, MA J, LIN J, et al. Unsupervised ship detection based on saliency and S-HOG descriptor from optical satellite images [J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(7): 1451-1455.
[5] MELGANI F, BRUZZONE L. Classification of hyperspectral remote sensing images with support vector machines [J]. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(8): 1778-1790.
[6] ZHOU Y M, WU Y R, SHEN X J, et al. Multi-kernel ensemble SVM for SAR target classification [J]. Command Information System and Technology, 2022, 13(3): 36-43 (in Chinese).
[7] KRISHNA L S, DATT K M, RAMESH Y R, et al. A novel framework for classfication of hyperspectral images [C]//2022 7th International Conference on Communication and Electronics Systems. Coimbatore: IEEE, 2022: 1704-1709.
[8] YANG W Q, LIU J, CHEN Z, et al. Recognition method of ship identification number based on deep learning [J]. Command Information System and Technology, 2022, 13(1): 58-63 (in Chinese).
[9] SUN H, ZHENG X T, LU X Q, et al. Spectral–spatial attention network for hyperspectral image classification [J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(5): 3232-3245.
[10] ROY S K, MANNA S, SONG T C, et al. Attention-based adaptive spectral–spatial kernel ResNet for hyperspectral image classification [J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(9): 7831-7843.
[11] GAO Z L, XIE J T, WANG Q L, et al. Global second-order pooling convolutional networks [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3019-3028.
[12] YANG Z X, ZHU L C, WU Y, et al. Gated channel transformation for visual recognition [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11791-11800.
[13] QIN Z Q, ZHANG P Y, WU F, et al. FcaNet: Frequency channel attention networks [C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 763-772.
[14] JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial transformer networks [C]// 29th Conference on Neural Information Processing Systems. Montreal: NIPS, 2015: 1-9.
[15] MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention [C]// 28th Conference on Neural Information Processing Systems. Montreal: NIPS, 2014: 1-9.
[16] ZHONG Y F, HU X, LUO C, et al. WHU-Hi: UAV-borne hyperspectral with high spatial resolution (H2) benchmark datasets and classifier for precise crop identification based on deep convolutional neural network with CRF [J]. Remote Sensing of Environment, 2020, 250: 112012.
[17] CHEN Y S, LIN Z H, ZHAO X, et al. Deep learning-based classification of hyperspectral data [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2094-2107.
[18] PAOLETTI M E, HAUT J M, FERNANDEZ-BELTRAN R, et al. Deep pyramidal residual networks for spectral–spatial hyperspectral image classification [J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(2): 740-754.