Unsupervised Regression Model of Geodesic Flow Kernel Based on Dynamic Independent Component Analysis and Dynamic Principal Component Analysis

doi:10.16183/j.cnki.jsjtu.2020.171

Abstract

Abstract:

It is difficult to accurately measure parameters by using the traditional soft sensor algorithm when the working condition of industrial process is changed. Therefore, a transfer learning strategy is introduced based on geodesic flow kernel to solve this problem. At the same time, the method is optimized to solve the problems of dynamic characteristic extraction and incomplete Gaussian distribution in industrial process. The augmented matrix is first constructed to deal with the dynamic characteristics of the process. Independent component analysis and principal component analysis are performed on the processed data to extract the non-Gaussian and Gaussian information of the source domain and the target domain. Then, the non-Gaussian and Gaussian information of the source domain is adapted to the target domain respectively on the Grassmann manifold. Finally, the maximum mean discrepancy is used to measure the distribution between the source domain and the target domain after domain adaptation, and the calculated results are applied to construct the weight of the model based on the source domain after domain adaptation. The results show that the method not only reduces the difference of distribution between the source domain and the target domain, but also solves the problems of dynamic characteristic extraction and incomplete Gaussian distribution in industrial process. The validity and the practicability of the model are proved by experiments on Tennessee Eastman data.

Key words: soft sensor, geodesic flow kernel, dynamic characteristics, dynamic independent component analysis, dynamic principal component analysis

CLC Number:

TP29

LAI Yanbo, YAN Gaowei, CHENG Lan, CHEN Zehua. Unsupervised Regression Model of Geodesic Flow Kernel Based on Dynamic Independent Component Analysis and Dynamic Principal Component Analysis[J]. Journal of Shanghai Jiao Tong University, 2020, 54(12): 1269-1277.

Figures/Tables 9

Fig.1

Fig.2

Tab.1

Fig.3

Tab.2

Fig.4

Tab.3

Fig.5

Fig.6

References 31

[1]	LI W T, WANG D H, CHAI T Y. Multisource data ensemble modeling for clinker free lime content estimate in rotary kiln sintering processes[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2015, 45(2): 303-314.
[2]	BIDAR B, SADEGHI J, SHAHRAKI F, et al. Data-driven soft sensor approach for online quality prediction using state dependent parameter models[J]. Chemometrics and Intelligent Laboratory Systems, 2017, 162: 130-141.
[3]	李浩，杨敏，石向荣，等. 基于主曲线的软测量方法及其在精馏塔上的应用[J]. 化工学报，2012, 63(8): 2492-2499.
	LI Hao, YANG Min, SHI Xiangrong, et al. Soft sensing method based on principal curves and its application in distillation columns[J]. Journal of Chemical Industry and Engineering (China), 2012, 63(8): 2492-2499.
[4]	颜学峰. 基于径基函数-加权偏最小二乘回归的干点软测量[J]. 自动化学报，2007, 33(2): 193-196.
	YAN Xuefeng. Radial basis function-weighted partial least square regression and its application to develop dry point soft sensor[J]. Acta Automatica Sinica, 2007, 33(2): 193-196.
[5]	KADLEC P, GRBIAC'G R, GABRYS B. Review of adaptation mechanisms for data-driven soft sensors[J]. Computers & Chemical Engineering, 2011, 35(1): 1-24.
[6]	杜永贵，李思思，阎高伟，等. 基于流形正则化域适应湿式球磨机负荷参数软测量[J]. 化工学报，2018, 69(3): 1244-1251.
	DU Yonggui, LI Sisi, YAN Gaowei, et al. Soft sensor of wet ball mill load parameter based on domain adaptation with manifold regularization[J]. Journal of Chemical Industry and Engineering (China), 2018, 69(3): 1244-1251.
[7]	ZHAO S J, ZHANG J, XU Y M.Performance monitoring of processes with multiple operating modes through multiple PLS models[J]. Journal of Process Control, 2006, 16(7): 763-772.
[8]	JIN H P, CHEN X G, WANG L, et al. Dual learning-based online ensemble regression approach for adaptive soft sensor modeling of nonlinear time-varying processes[J]. Chemometrics and Intelligent Laboratory Systems, 2016, 151: 228-244.
[9]	李元，张新民. 基于非高斯信息的JITL软测量模型[J]. 上海交通大学学报，2015, 49(6): 897-901.
	LI Yuan, ZHANG Xinmin. Non-Gaussian information based JITL soft sensor model[J]. Journal of Shanghai Jiao Tong University, 2015, 49(6): 897-901.
[10]	DAUME H, MARCU D. Domain adaptation for statistical classifiers[J]. Journal of Artificial Intelligence Research, 2011, 26(1): 101-126.
[11]	LV X, GUAN Y, DENG B Y. Transfer learning based clinical concept extraction on data from multiple sources[J]. Journal of Biomedical Informatics, 2014, 52: 55-64.
[12]	SUN S L, SHI H L, WU Y B. A survey of multi-source domain adaptation[J]. Information Fusion, 2015, 24: 84-92.
[13]	CHEN H Y, CHIEN J T. Deep semi-supervised learning for domain adaptation[C]∥25th International Workshop on Machine Learning for Signal Processing (MLSP). Boston, MA, USA: IEEE, 2015: 1-6.
[14]	FERNANDO B, TOMMASI T, TUYTELAARS T. Joint cross-domain classification and subspace learning for unsupervised adaptation[J]. Pattern Recognition Letters, 2015, 65: 60-66.
[15]	WANG J T, ZHENG H, HUANG Y, et al. Vehicle type recognition in surveillance images from labeled web-nature data using deep transfer learning[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(9): 2913-2922.
[16]	AFZAL M Z, KÖLSCH A, AHMED S, et al. Cutting the error by half: Investigation of very deep cnn and advanced training strategies for document image classification[C]∥14th IAPR International Conference on Document Analysis and Recognition (ICDAR). Kyoto, Japan: IEEE, 2017, 1: 883-888.
[17]	XU B H, FU Y W, JIANG Y G, et al. Heterogeneous knowledge transfer in video emotion recognition, attribution and summarization[J]. IEEE Transactions on Affective Computing, 2018, 9(2): 255-270.
[18]	LIU Y, YANG C, LIU K X, et al. Domain adaptation transfer learning soft sensor for product quality prediction[J]. Chemometrics and Intelligent Laboratory Systems, 2019, 192: 103813.
[19]	贺敏，汤健，郭旭琦，等．基于流形正则化域适应随机权神经网络的湿式球磨机负荷参数软测量[J]. 自动化学报，2019, 45(2): 398-406.
	HE Min, TANG Jian, GUO Xuqi, et al. Soft sensor for ball mill load using DAMRRWNN model[J]. Acta Automatica Sinica, 2019, 45(2): 398-406.
[20]	GOPALAN R, LI R, CHELLAPPA R. Domain adaptation for object recognition: An unsupervised approach[C]∥International Conference On Computer Vision. Barcelona, Spain: IEEE, 2011: 999-1006.
[21]	GONG B Q, SHI Y, SHA F, et al. Geodesic flow kernel for unsupervised domain adaptation[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE, 2012: 2066-2073.
[22]	SAMAT A, GAMBA P, ABUDUWAILI J, et al. Geodesic flow kernel support vector machine for hyperspectral image classification by unsupervised subspace feature transfer[J]. Remote Sensing, 2016, 8(3): 234.
[23]	KUMAR S, SAVAKIS A. Robust domain adaptation on the l1-grassmannian manifold[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Las Vegas, NV, USA: IEEE, 2016: 103-110.
[24]	KU W F, STORER R H, GEORGAKIS C. Disturbance detection and isolation by dynamic principal component analysis[J]. Chemometrics and Intelligent Laboratory Systems, 1995, 30(1): 179-196.
[25]	LEE J M, YOO C K, LEE I B. Statistical monitoring of dynamic processes based on dynamic independent component analysis[J]. Chemical Engineering Science, 2004, 59(14): 2995-3006.
[26]	ZHANG G M, LI N, LI S Y. Data-driven process monitoring method based on dynamic component analysis[C]∥Proceedings of the 30th Chinese Control Conference. Yantai, China: IEEE, 2011: 5288-5293.
[27]	STEFATOS G, HAMZA A B. Dynamic independent component analysis approach for fault detection and diagnosis[J]. Expert Systems with Applications, 2010, 37(12): 8606-8617.
[28]	GONG B Q, GRAUMAN K, SHA F. Learning kernels for unsupervised domain adaptation with applications to visual object recognition[J]. International Journal of Computer Vision, 2014, 109: 3-27.
[29]	GRETTON A, BORGWARDT K, RASCH M, et al. A kernel method for the two-sample-problem[C]∥Advances In Neural Information Processing Systems. Vancouver, British Columbia, Canada: NIPS, 2006: 513-520.
[30]	BATHELT A, RICKER N L, JELALI M.Revision of the tennessee eastman process model[J]. IFAC-PapersOnLine, 2015, 48(8): 309-314.
[31]	张淑美，王福利，谭帅，等. 多模态过程的全自动离线模态识别方法[J]. 自动化学报，2016, 42(1): 60-80.
	ZHANG Shumei, WANG Fuli, TAN Shuai, et al. A fully automatic offline mode identification method for multi-mode processes[J]. Acta Automatica Sinica, 2016, 42(1): 60-80.

预测标签	方法	RMSE值
预测标签	方法	工况2	工况3	工况4	工况5
物料A	PLSR	3.665 2	2.798 1	1.937 8	1.592 3
	GFK	0.736 8	1.607 9	1.043 8	0.564 5
	ICA-GFK	1.477 9	1.852 5	1.105 9	1.228 7
物料C	PLSR	3.753 2	3.152 9	4.457 7	4.468 8
	GFK	2.403 7	2.872 1	2.413 5	1.427 9
	ICA-GFK	2.540 3	2.829 2	2.511 8	1.950 7
物料H	PLSR	1.064 0	1.474 3	1.783 9	2.786 8
	GFK	0.665 1	0.549 5	0.472 7	0.543 9
	ICA-GFK	0.770 8	0.555 4	0.459 6	0.535 7

预测标签	方法	RMSE值
预测标签	方法	工况2	工况3	工况4	工况5
物料A	GFK	0.736 8	1.607 9	1.043 8	0.564 5
	DPCA-GFK	0.726 2	1.582 4	1.006 1	0.542 5
	DICA-DPCA-GFK	0.705 5	1.457 5	0.872 6	0.463 7
物料C	GFK	2.403 7	2.872 1	2.413 5	1.427 9
	DPCA-GFK	2.397 8	2.823 2	2.390 2	1.426 4
	DICA-DPCA-GFK	2.343 5	2.754 2	2.284 1	1.163 9
物料H	GFK	0.665 1	0.549 5	0.472 7	0.543 9
	DPCA-GFK	0.648 9	0.535 9	0.424 5	0.495 8
	DICA-DPCA-GFK	0.644 0	0.495 5	0.388 7	0.487 6

预测标签	方法	RMSE值
预测标签	方法	工况2	工况3	工况4	工况5
物料A	ICA-GFK	1.477 9	1.852 5	1.105 9	1.228 7
	DICA-GFK	1.438 7	1.783 8	1.059 4	1.094 5
	DICA-DPCA-GFK	0.705 5	1.457 5	0.872 6	0.463 7
物料C	ICA-GFK	2.540 3	2.829 2	2.511 8	1.950 7
	DICA-GFK	2.401 4	2.815 6	2.416 2	1.821 9
	DICA-DPCA-GFK	2.343 5	2.754 2	2.284 1	1.163 9
物料H	ICA-GFK	0.770 8	0.555 4	0.459 6	0.535 7
	DICA-GFK	0.733 3	0.541 8	0.466 4	0.503 5
	DICA-DPCA-GFK	0.644 0	0.495 5	0.388 7	0.487 6