Prediction Method of Equipment Remaining Life Based on Self-Attention Long Short-Term Memory Neural Network

doi:10.1007/s12204-022-2506-6

Abstract

Abstract: : Aiming at the problem of insufficient consideration of the correlation between components in the prediction of the remaining life of mechanical equipment, the method of remaining life prediction that combines the self-attention mechanism with the long short-term memory neural network (LSTM-NN) is proposed, called Self-Attention-LSTM. First, the auto-encoder is used to obtain the component-level state information; second, the state information of each component is input into the self-attention mechanism to learn the correlation between components; then, the multi-component correlation matrix is added to the LSTM input gate, and the LSTM-NN is used for life prediction. Finally, combined with the commercial modular aero-propulsion system simulation data set (C-MAPSS), the experiment was carried out and compared with the existing methods. Research results show that the proposed method can achieve better prediction accuracy and verify the feasibility of the method.

Key words: equipment remaining life prediction, self-attention, long short-term memory neural network (LSTMNN), correlation analysis

摘要： 针对机械设备剩余寿命预测中部件间相关性影响考虑不充分的问题，提出一种结合自注意力机制(Self-Attention)与长短期记忆神经网络(LSTM-NN)的剩余寿命预测方法，Self-Attention-LSTM。首先，利用自编码器获取部件级状态信息；其次，将各部件状态信息输入自注意力机制进行部件间关联度学习；然后，多部件关联性矩阵加入LSTM输入门中，运用长短期记忆神经网络进行寿命预测。最后，结合航空发动机系统仿真数据集(C-MAPSS)展开试验并与已有方法进行对比分析，研究结果表明提出方法能够达到较好的预测精度，验证了方法的可行性。

关键词: 设备剩余寿命预测，自注意力机制，长短期记忆神经网络（LSTM-NN），关联性分析

CLC Number:

TH17

CAO Xiangang1,2(曹现刚)，LEI Zhuol*(雷卓)，LI Yanchuan1(李彦川), ZHANG Mengyuan1(张梦园),DUAN Xinyu1(段欣宇). Prediction Method of Equipment Remaining Life Based on Self-Attention Long Short-Term Memory Neural Network[J]. J Shanghai Jiaotong Univ Sci, 2023, 28(5): 652-664.

References

[1] COMPARE M, BELLANI L, ZIO E. Reliability model of a component equipped with PHM capabilities [J]Reliability Engineering & System Safety, 2017, 168: 4-11.
[2] CHEN Z, CHEN X, DE OLIVIRA J V. Application of deep learning in equipment prognostics and health management [J]. Chinese Journal of Scientific Instrument, 2019, 40(9): 206-226 (in Chinese).
[3] LIAO L X, K?TTIG F. A hybrid framework combining data-driven and model-based methods for system remaining useful life prediction [J]. Applied Soft Computing, 2016, 44: 191-199.
[4] WANG Y D, ZHAO Y F, ADDEPALLI S. Remaining useful life prediction using deep learning approaches: A review [J]. Procedia Manufacturing, 2020, 49: 81-88.
[5] AL-DAYIAN G R, EL-HELBAWY A A, REFAEY R M, et al. Bayesian estimation and prediction based on constant stress-partially accelerated life testing for topp leone-inverted kumaraswamy distribution [J]. Journal of Advances in Mathematics and Computer Science, 2021, 36(3): 11-32.
[6] LIN J D, LIAO G B, CHEN M, et al. Two-phase degradation modeling and remaining useful life prediction using nonlinear Wiener process [J]. Computers & Industrial Engineering, 2021, 160: 107533.
[7] CHEN J, YUAN S F, SBARUFATTI C, et al. Dual crack growth prognosis by using a mixture proposal particle filter and on-line crack monitoring [J]. Reliability Engineering & System Safety, 2021, 215: 107758.
[8] WANG X L, BALAKRISHNAN N, GUO B. Residual life estimation based on nonlinear-multivariate Wiener processes [J]. Journal of Statistical Computation and Simulation, 2015, 85(9): 1742-1764.
[9] ZHANG H W, CHEN M Y, ZHOU D H. Remaining useful life prediction for a nonlinear multi-degradation system with public noise [J]. Journal of Systems Engineering and Electronics, 2018, 29(2): 429-435.
[10] LIN W J. Particle filtering-based degradation modeling and remaining useful life prediction for muticomponent systems [D]. Chengdu: University of Electronic Science and Technology of China, 2019 (in Chinese).
[11] BAI C, HU C H, ZHANG J X, et al. Life prediction method of multi-component degradation equipment with the impact of random shocks [J]. China Measurement & Test, 2018, 44(7): 124-131 (in Chinese).
[12] YANG Y, ZHANG N, CHENG J S. Global parameters dynamic learning deep belief networks and its application in rolling bearing life prediction [J]. Journal of Vibration and Shock, 2019, 38(10): 199-205, 249 (in Chinese).
[13] WANG Y J, WANG S D, KANG S Q, et al. prediction method of remaining useful life of rolling bearings based on improved GcForest [J]. Proceedings of the CSEE, 2020, 40(15): 5032-5042 (in Chinese).
[14] LIPTON Z C, BERKOWITZ J, ELKAN C. A critical review of recurrent neural networks for sequence learning [DB/OL]. (2015-10-17). https://arxiv.org/abs/1506.00019.
[15] YU W N, KIM I Y, MECHEFSKE C. An improved similarity-based prognostic algorithm for RUL estimation using an RNN autoencoder scheme [J]. Reliability Engineering & System Safety, 2020, 199: 106926.
[16] PEI H, HU C H, SI X S, et al. Review of machine learning based remaining useful life prediction methods for equipment [J]. Journal of Mechanical Engineering, 2019, 55(8): 1-13 (in Chinese).
[17] PENG K X, PI Y T, JIAO R H, et al. Health indicator construction and remaining useful life prediction for aircraft engine [J]. Control Theory & Applications, 2020, 37(4): 713-720 (in Chinese).
[18] ZHANG J D, ZOU Y S, DENG J L, et al. Bearing remaining life prediction based on full convolutional layer neural networks [J]. China Mechanical Engineering, 2019, 30(18): 2231-2235 (in Chinese).
[19] GREFF K, SRIVASTAVA R K, KOUTNIK J, et al. LSTM: A search space odyssey [J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2222-2232.
[20] SATEESH BABU G, ZHAO P L, LI X L. Deep convolutional neural network based regression approach for estimation of remaining useful life [M]//Database systems for advanced applications. Cham: Springer, 2016: 214-228.
[21] SONG Y, XIA T, ZHENG Y, et al. A remaining useful life prediction framework integrating multiple time window convolutional neural networks [J]. The Journal of Grey System, 2020, 32(3): 34-48.
[22] TANG X, XU W X, TAN J W, et al. Prediction for remaining useful life of rolling bearings based on Long Short-Term Memory [J]. Journal of Machine Design, 2019, 36(S1): 117-119 (in Chinese).
[23] ZHANG J J, WANG P, YAN R Q, et al. Long shortterm memory for machine remaining life prediction [J]. Journal of Manufacturing Systems, 2018, 48: 78-86.
[24] CHEN J L, JING H J, CHANG Y H, et al. Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process [J]. Reliability Engineering & System Safety, 2019, 185: 372-382.
[25] SONG Y, SHI G, CHEN L Y, et al. Remaining useful life prediction of turbofan engine using hybrid model based on autoencoder and bidirectional long shortterm memory [J]. Journal of Shanghai Jiao Tong University (Science), 2018, 23(Sup.1): 85-94.
[26] LI J F, CHEN Y X, XIANG H C, et al. Remaining useful life prediction for aircraft engine based on LSTMDBN [J]. Systems Engineering and Electronics, 2020, 42(7): 1637-1644.
[27] HU C H, HU C H, SI X S, et al. Remaining useful lifetime prediction method of rolling bearing based on MSCNN-LSTM [J]. China Measurement & Testing Technology, 2020, 46(9): 103-110 (in Chinese).
[28] LU J X, ZHANG Q P, YANG Z H, et al. Shortterm load forecasting method based on CNN-LSTM hybrid neural network model [J]. Automation of Electric Power Systems, 2019, 43(8): 131-137 (in Chinese).[29] ZHANG Y F, LU Z Q. Remaining useful life prediction based on an integrated neural network [J]. Chinese Journal of Engineering, 2020, 42(10): 1372-1380 (in Chinese).
[30] XIANG S, QIN Y, ZHU C C, et al. Long short-term memory neural network with weight amplification and its application into gear remaining useful life prediction [J]. Engineering Applications of Artificial Intelligence, 2020, 91: 103587.
[31] XIA T B, SONG Y, ZHENG Y, et al. An ensemble framework based on convolutional bi-directional LSTM with multiple time windows for remaining useful life estimation [J]. Computers in Industry, 2020, 115: 103182.
[32] CHEN Y H, PENG G L, ZHU Z Y, et al. A novel deep learning method based on attention mechanism for bearing remaining useful life prediction [J]. Applied Soft Computing, 2020, 86: 105919.
[33] QIN Y, XIANG S, CHAI Y, et al. Macroscopic– microscopic attention in LSTM networks based on fusion features for gear remaining life prediction [J]. IEEE Transactions on Industrial Electronics, 2020, 67(12): 10865-10875.
[34] ZHAO R, YAN R Q, CHEN Z H, et al. Deep learning and its applications to machine health monitoring [J]. Mechanical Systems and Signal Processing, 2019, 115: 213-237.
[35] SHAO Q, MA H P. Convolutional neural network text classification model with self-attention mechanism [J]. Journal of Chinese Computer Systems, 2019, 40(6): 1137-1141 (in Chinese).
[36] GAO X J, ZHANG Z, MU T T, et al. Self-attention driven adversarial similarity learning network [J]. Pattern Recognition, 2020, 105: 107331.
[37] HOCHREITER S, SCHMIDHUBER J. Long shortterm memory [J]. Neural Computation, 1997, 9(8): 1735-1780.
[38] ZHENG S, RISTOVSKI K, FARAHAT A, et al. Long short-term memory network for remaining useful life estimation [C]//2017 IEEE International Conference on Prognostics and Health Management. Dallas: IEEE, 2017: 88-95.
[39] ELSHEIKH A, YACOUT S, OUALI M S. Bidirectional handshaking LSTM for remaining useful life prediction [J]. Neurocomputing, 2019, 323: 148-156.
[40] LI X, DING Q, SUN J Q. Remaining useful life estimation in prognostics using deep convolution neural networks [J]. Reliability Engineering & System Safety, 2018, 172: 1-11.
[41] SAXENA A, GOEBEL K, SIMON D, et al. Damage propagation modeling for aircraft engine runto-failure simulation [C]//2008 International Conference on Prognostics and Health Management. Denver: IEEE, 2008: 1-9.
[42] DENG K Y, ZHANG X Y, CHENG Y J, et al. A remaining useful life prediction method with long-short term feature processing for aircraft engines [J]. Applied Soft Computing, 2020, 93: 106344.
[43] SUTHARSSAN T, STOYANOV S, BAILEY C, et al. Prognostic and health management for engineering systems: A review of the data-driven approach and algorithms [J]. The Journal of Engineering, 2015, 2015(7): 215-222.