Person Re-Identification Based on Spatial Feature Learning and Multi-Granularity Feature Fusion

doi:10.1007/s12204-023-2626-7

Abstract

Abstract: In view of the weak ability of the convolutional neural networks to explicitly learn spatial invariance and the probabilistic loss of discriminative features caused by occlusion and background interference in pedestrian re-identification tasks, a person re-identification method combining spatial feature learning and multi-granularity feature fusion was proposed. First, an attention spatial transformation network (A-STN) is proposed to learn spatial features and solve the problem of misalignment of pedestrian spatial features. Then the network was divided into a global branch, a local coarse-grained fusion branch, and a local fine-grained fusion branch to extract pedestrian global features, coarse-grained fusion features, and fine-grained fusion features, respectively. Among them, the global branch enriches the global features by fusing different pooling features. The local coarse-grained fusion branch uses an overlay pooling to enhance each local feature while learning the correlation relationship between multi-granularity features. The local fine-grained fusion branch uses a differential pooling to obtain the differential features that were fused with global features to learn the relationship between pedestrian local features and pedestrian global features. Finally, the proposed method was compared on three public datasets: Market1501, DukeMTMC-ReID and CUHK03. The experimental results were better than those of the comparative methods, which verifies the effectiveness of the proposed method.

Key words: pedestrian re-identification, spatial features, attention spatial transformation network, multi-branch network, relation features

摘要： 针对卷积神经网络显式学习空间不变性的能力较弱以及行人重识别任务中存在遮挡、背景干扰等导致辨别性特征概率性丢失的问题，提出了一种结合空间特征学习与多粒度特征融合的行人重识别方法。首先，提出注意力空间转换网络(A-STN)，用于学习空间特征，解决行人空间特征未对齐的问题；然后，将网络分为全局分支、局部粗粒度融合分支和局部细粒度融合分支，分别提取行人全局特征、粗粒度融合特征和细粒度融合特征。其中全局分支通过融合不同池化特征丰富全局特征；局部粗粒度融合分支利用重叠池化增强每个局部特征的同时学习多粒度特征之间的关联关系；局部细粒度融合分支则通过利用差异化池化，得到的差异特征，与全局特征进行相互融合，以学习行人局部特征和全局特征之间的关联关系。所提出的方法在Market1501、DukeMTMC-ReID和CUHK03三个公开数据集上开展对比实验，实验结果均优于所对比的方法，验证了所提方法的有效性。

关键词: 行人重识别；空间特征；注意力空间转换网络；多分支网络；关联特征

CLC Number:

Diao Zijian, Cao Shuai, Li Wenwei, Liang Jianan, Wen Guilin, Huang Weixi, Zhang Shouming. Person Re-Identification Based on Spatial Feature Learning and Multi-Granularity Feature Fusion[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(2): 363-374.

References

[1] LUO H, JIANG W, FAN X, et al. A survey on deep learning based person re-identification [J]. Acta Automatica Sinica, 2019, 45(11): 2032-2049 (in Chinese).
[2] LIAO S C, HU Y, ZHU X Y, et al. Person reidentification by Local Maximal Occurrence representation and metric learning [C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 2197-2206.
[3] ZHAO R, OUYANGW L, WANG X G. Learning midlevel filters for person re-identification [C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 144-151.
[4] SUN Y F, ZHENG L, YANG Y, et al. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline) [M]//Computer vision—ECCV 2018. Cham: Springer, 2018: 501-518.
[5] ZHENG L, HUANG Y J, LU H C, et al. Pose invariant embedding for deep person re-identification [J]. IEEE Transactions on Image Processing, 2019, 28(9): 4500- 4509.
[6] WANG G S, YUAN Y F, CHEN X, et al. Learning discriminative features with multiple granularities for person re-identification [C]//26th ACM international conference on Multimedia. Seoul: ACM, 2018: 274- 282.
[7] ZHENG Z D, ZHENG L, YANG Y. Pedestrian alignment network for large-scale person re-identification [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29(10): 3037-3045.
[8] ZHAO H Y, TIAN M Q, SUN S Y, et al. Spindle net: Person re-identification with human body region guided feature decomposition and fusion [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: 2017: 907-915.
[9] PENG P X, TIAN Y H, HUANG Y R, et al. Discriminative spatial feature learning for person reidentification [C]//Proceedings of the 28th ACM International Conference on Multimedia. Seattle: ACM, 2020: 274-283.
[10] BAI X R, HUI Y, WANG L, et al. Radar-based human gait recognition using dual-channel deep convolutional neural network [J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(12): 9767-9778.
[11] CHAKRABORTY M, KUMAWAT H C, DHAVALE S V, et al. DIAT-RadHARNet: A lightweight DCNN for radar based classification of human suspicious activities [J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-10.
[12] HE KM, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[13] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2818- 2826.
[14] CHEN W H, CHEN X T, ZHANG J G, et al. Beyond triplet loss: A deep quadruplet network for person re-identification [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1320-1329.
[15] JADERBERG M, SIMONYAN K, ZISSERMAN A. Spatial transformer networks [C]//Advances in Neural Information Processing Systems. Montreal: NIPS, 2015: 1-9.
[16] HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[17] PARK H, HAM B. Relation network for person reidentification [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11839-11847.
[18] ZHENG L, SHEN L Y, TIAN L, et al. Scalable person re-identification: A benchmark [C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1116-1124.
[19] RISTANI E, SOLERA F, ZOU R, et al. Performance measures and a data set for multi-target, multi-camera tracking [M]//Computer vision—ECCV 2016 Workshops. Cham: Springer, 2016: 17-35.
[20] LI W, ZHAO R, XIAO T, et al. DeepReID: deep filter pairing neural network for person re-identification [C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 152-159.
[21] RISTANI E, TOMASI C. Features for multi-target multi-camera tracking and re-identification [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6036- 6046.
[22] LUO H, GU Y Z, LIAO X Y, et al. Bag of tricks and a strong baseline for deep person re-identification [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach: IEEE, 2019: 1487-1495.
[23] CHEN F, WANG N, TANG J, et al. A feature disentangling approach for person re-identification via selfsupervised data augmentation [J]. Applied Soft Computing, 2021, 100: 106939.
[24] ZHENG F, DENG C, SUN X, et al. Pyramidal person re-IDentification via multi-loss dynamic training [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 8506-8514.
[25] DAI Z Z, CHEN M Q, GU X D, et al. Batch Drop- Block network for person re-identification and beyond [C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3690-3700.
[26] WANG C, ZHANG Q, HUANG C, et al. Mancs: A multi-task attentional network with curriculum sampling for person re-identification [M]//Computer vision—ECCV 2018. Cham: Springer, 2018: 384-400.
[27] LI W, ZHU X T, GONG S G. Harmonious attention network for person re-identification [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 2285- 2294.
[28] ZHENG M, KARANAM S, WU Z Y, et al. Reidentification with consistent attentive Siamese networks [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5728-5737.
[29] LUO H, JIANG W, FAN X, et al. STNReID: Deep convolutional networks with pairwise spatial transformer networks for partial person re-identification [J]. IEEE Transactions on Multimedia, 2020, 22(11): 2905-2913.
[30] QUISPE R, PEDRINI H. Top-DB-net: Top DropBlock for activation enhancement in person re-identification [C]//2020 25th International Conference on Pattern Recognition. Milan: IEEE, 2021: 2980-2987.
[31] ZHU K, GUO H Y, ZHANG S L, et al. AAformer: Auto-aligned transformer for person re-identification [DB/OL]. (2021-04-02). https://arxiv.org/abs/ 2104.00921

[32] TANG Y Z, YANG X, WANG N N, et al. Person reidentification with feature pyramid optimization and gradual background suppression [J]. Neural Networks, 2020, 124: 223-232.
[33] LI Y L, HE J F, ZHANG T Z, et al. Diverse part discovery: Occluded person re-identification with part-aware transformer [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 2897-2906.
[34] WANG Z K, ZHU F, TANG S X, et al. Feature erasing and diffusion network for occluded person re-identification [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 4744-4753.