J Shanghai Jiaotong Univ Sci

Journal of Shanghai Jiaotong University(Science) >

2025 , Vol. 30 >Issue 5: 952 - 961

DOI: https://doi.org/10.1007/s12204-024-2699-y

Computing & Computer Technologies

Hypergraph-Based Asynchronous Event Processing for Moving Object Classification

Expand
  • 1. Key Laboratory of Social Computing and Cognitive Intelligence of Ministry of Education; School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, Liaoning, China; 2. HiSilicon (Shanghai) Technologies Co, Ltd., Shanghai 200040, China

Received date: 2023-08-03

  Accepted date: 2023-08-24

  Online published: 2024-01-16

Fold

Abstract

Unlike traditional video cameras, event cameras capture asynchronous event streams in which each event encodes pixel location, triggers’ timestamps, and the polarity of brightness changes. In this paper, we introduce a novel hypergraph-based framework for moving object classification. Specifically, we capture moving objects with an event camera, to perceive and collect asynchronous event streams in a high temporal resolution. Unlike stacked event frames, we encode asynchronous event data into a hypergraph, fully mining the high-order correlation of event data, and designing a mixed convolutional hypergraph neural network for training to achieve a more efficient and accurate motion target recognition. The experimental results show that our method has a good performance in moving object classification (e.g., gait identification).

Cite this article

YU Nannan, WANG Chaoyi, QIAO Yu, WANG Yuxin, ZHENG Chenglin, ZHANG Qiang, YANG Xin . Hypergraph-Based Asynchronous Event Processing for Moving Object Classification[J]. Journal of Shanghai Jiaotong University(Science), 2025 , 30(5) : 952 -961 . DOI: 10.1007/s12204-024-2699-y

References

[1]    LI J, TIAN Y. RECent advances in neuromorphic vision sensors: A survey [J]. Chinese Journal of Computers: 2021, 44(6): 1258-1286 (in Chinese).

[2] ERNST M O, BANKS M S. Humans integrate visual and haptic information in a statistically optimal fashion [J]. Nature, 2002, 415(6870): 429-433.

[3] HAN J, BHANU B. Individual recognition using gait energy image [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(2): 316-322.

[4] LEE L, GRIMSON W E L. Gait analysis for recognition and classification [C]//Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition. Washington: IEEE, 2002: 155-162.

[5] WANG L, TAN T N, NING H Z, et al. Silhouette analysis-based gait recognition for human identification [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(12): 1505-1518.

[6] TAO D C, LI X L, WU X D, et al. General tensor discriminant analysis and Gabor features for gait recognition [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(10): 1700-1715.

[7] WU Z F, HUANG Y Z, WANG L, et al. A comprehensive study on cross-view gait based human identification with deep CNNs [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(2): 209-226.

[8] SHIRAGA K, MAKIHARA Y, MURAMATSU D, et al. GEINet: View-invariant gait recognition using a convolutional neural network [C]//2016 International Conference on Biometrics. Halmstad: IEEE, 2016: 1-8.

[9] WOLF T, BABAEE M, RIGOLL G. Multi-view gait recognition using 3D convolutional neural networks [C]//2016 IEEE International Conference on Image Processing. Phoenix: IEEE, 2016: 4165-4169.

[10] ALOTAIBI M, MAHMOOD A. Improved gait recognition based on specialized deep convolutional neural network [J]. Computer Vision and Image Understanding, 2017, 164: 103-110.

[11] LIU Z Y, SARKAR S. Simplest representation yet for gait recognition: Averaged silhouette [C]//Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR. Cambridge: IEEE, 2004: 211-214.

[12] NILSBACK M E, ZISSERMAN A. Automated flower classification over a large number of classes [C]//2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing. Bhubaneswar: IEEE, 2008: 722-729.

[13] AMIR A, TABA B, BERG D, et al. A low power, fully event-based gesture recognition system [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 7388-7397.

[14] LAGORCE X, ORCHARD G, GALLUPPI F, et al. HOTS: A hierarchy of event-based time-surfaces for pattern recognition [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(7): 1346-1359.

[15] PARK P K J, LEE K, LEE J H, et al. Computationally efficient, real-time motion recognition based on bio-inspired visual and cognitive processing [C]//2015 IEEE International Conference on Image Processing. Quebec City: IEEE, 2015: 932-935.

[16] BI Y, CHADHA A, ABBAS A, et al. Graph-based spatio-temporal feature learning for neuromorphic vision sensing [J]. IEEE Transactions on Image Processing, 2020, 29: 9084-9098.

[17] GAO G, KYRARINI M, RAZAVI M, et al. Comparison of Dynamic Vision Sensor-Based and IMU-based systems for ankle joint angle gait analysis [C]//2016 2nd International Conference on Frontiers of Signal Processing. Warsaw: IEEE, 2016: 93-98.

[18] LI Y J, ZHOU H, YANG B B, et al. Graph-based asynchronous event processing for rapid object recognition [C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 914-923.

[19] SCHAEFER S, GEHRIG D, SCARAMUZZA D. AEGNN: asynchronous event-based graph neural networks [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 12361-12371.

[20] WANG Y X, ZHANG X, SHEN Y R, et al. Event-stream representation for human gaits identification using deep neural networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3436-3449.

[21]  BERGE C. Hypergraphs: combinatorics of finite sets[M]. Amsterdam: Elsevier, 1984.

[22] KAHNG A B. Fast hypergraph partition [C]//Proceedings of the 26th ACM/IEEE Design Automation Conference. Las Vegas: ACM, 1989: 762-766.

[23] ALON N. Transversal numbers of uniform hypergraphs [J]. Graphs and Combinatorics, 1990, 6(1): 1-4.

[24] GUNOPULOS D, MANNILA H, KHARDON R, et al. Data mining, hypergraph transversals, and machine learning (extended abstract) [C]//Sixteenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. Tucson: ACM, 1997: 209-216.

[25] CHUNG F. The Laplacian of a hypergraph [M]//Expanding graphs. Providence: AMS, 1992: 21-36.

[26] EITER T, GOTTLOB G. Identifying the minimal transversals of a hypergraph and related problems [J]. SIAM Journal on Computing, 1995, 24(6): 1278-1304.

[27] KARYPIS G, AGGARWAL R, KUMAR V, et al. Multilevel hypergraph partitioning: Applications in VLSI domain [J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 1999, 7(1): 69-79.

[28] BULÒ S R, PELILLO M. A game-theoretic approach to hypergraph clustering [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 35: 1312-1327.

[29] PAPA D, MARKOV I. Hypergraph partitioning and clustering [M]// Handbook of approximation algorithms and metaheuristics. Boca Raton: Chapman and Hall/CRC, 2007: 61.

[30] SUN L, JI S W, YE J P. Hypergraph spectral learning for multi-label classification [C]// 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Las Vegas: ACM, 2008: 668-676.

[31] ZASS R, SHASHUA A. Probabilistic graph and hypergraph matching [C]//2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage: IEEE, 2008: 1-8.

[32] BRETTO A, GILLIBERT L. Hypergraph-based image representation [M]//Graph-based representations in pattern recognition. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005: 1-11.

[33] GUI H, LIU J L, TAO F B, et al. Large-scale embedding learning in heterogeneous event data [C]//2016 IEEE 16th International Conference on Data Mining. Barcelona: IEEE, 2016: 907-912.

[34] TU K, CUI P, WANG X A, et al. Structural deep embedding for hyper-networks [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): 426-433.

[35] ZHANG R C, ZOU Y S, MA J. Hyper-SAGNN: A self-attention based graph neural network for hypergraphs [DB/OL]. (2019-11-06). https://arxiv.org/abs/1911.0261

[36] YADATI N, NIMISHAKAVI M, YADAV P, et al. HyperGCN: Hypergraph convolutional networks for semi-supervised learning and combinatorial optimisation [DB/OL]. (2018-09-07). https://arxiv.org/abs/1809.02589

[37] STOFFREGEN T, GALLEGO G, DRUMMOND T, et al. Event-based motion segmentation by motion compensation [C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 7243-7252.

[38] ALONSO I, MURILLO A C. EV-SegNet: Semantic segmentation for event-based cameras [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach: IEEE, 2019: 1624-1633.

[39] MITROKHIN A, HUA Z Y, FERMÜLLER C, et al. Learning visual motion segmentation using event surfaces [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 14402-14411.

[40] BAI S, ZHANG F H, TORR P H S. Hypergraph convolution and hypergraph attention [J]. Pattern Recognition, 2021, 110: 107637.

[41] AGARWAL S, BRANSON K, BELONGIE S. Higher order learning with graphs [C]// 23rd International Conference on Machine Learning. Pittsburgh: ACM, 2006: 17-24.

[42] BI Y, CHADHA A, ABBAS A, et al. Graph-based object classification for neuromorphic vision sensing [C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 491-501.

Options
Outlines

/

〈 〉