Fast Leave-One-Out Cross-Validation Algorithm for Extreme Learning Machine

Abstract

Abstract: Leaveoneout crossvalidation has proved to be near capable of giving the unbiased estimation of the generalization performance of statistical models, and thus can provide a reliable criterion for model selection and comparison. For this reason, the current paper presented a fast leaveoneout crossvalidation algorithm in the framework of extreme learning machines (ELMs) with respect to both regression and classification problems, which can avoid training explicitly and just has the complexity of O(N) for a data set with N points. The validity of the algorithm is also strictly proved. The simulations conducted on the artificial and realworld problems show the effectiveness and efficiency of the proposed algorithm.

Key words: extreme learning machine (ELM), leaveoneout, crossvalidation, computational complexity

CLC Number:

TP18
O234

LIU Xue-Yi-a, LI Ping-a, b , GAO Chuan-Hou-c. Fast Leave-One-Out Cross-Validation Algorithm for Extreme Learning Machine[J]. Journal of Shanghai Jiaotong University, 2011, 45(08): 1140-1145.

References

［1］Xu J X, Hou Z S. Notes on datadriven system approaches［J］. Acta Automatics Sinica, 2009, 35(6): 668675.

［2］许开州，胡德金，魏臣隽. 基于正交试验和Vogl快速BP网络的球面磨削工艺优化方法［J］. 上海交通大学学报，2009，43（12）：19561961.

XU Kaizhou, HU Dejin, WEI Chenjun. Vogl fast BP network and orthogonal experiment method in optimization of sphere grinding process parameters［J］. Journal of Shanghai Jiaotong University, 2009, 43(12): 19561961.

［3］Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: Theory and applications［J］. Neurocomputing, 2006, 70: 489501.

［4］Zhang R, Huang G B, Sundararajan N, et al. Multicategory classification using extreme learning machine for microarray gene expression cancer diagnosis［J］. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2007, 4(3): 185495.

［5］Miche Y, Sorjamaa A, Bas P, et al. OPELM: Optimally pruned extreme learning machine［J］. IEEE Transactions on Neural Networks,2010,21(1):159162.

［6］Kim J, Shin H S, Shin K, et al. Robust algorithm for arrhythmia classification in ECG using extreme learning machine［J］. BioMedical Engineering Online, 2009, 8: 112.

［7］邓万宇，郑庆华，陈琳，等. 神经网络极速学习方法研究［J］. 计算机学报，2010，33（2）：279287.

DENG Wanyu, ZHENG Qinghua, CHEN Lin, et al. Research on extreme learning of neural networks［J］. Chinese Journal of Computers, 2010, 33(2): 279287.

［8］Opper M, Winter O. Gaussian processes and SVM: Mean field and leaveone out［C］//Advances in Large Margin Classifers. Cambridge: MIT Press, 2000: 311326.

［9］Zhao Y, Kwoh C K. Fast leaveoneout evaluation and improvement on inference for LSSVMs［C］//Proceedings of the 17th International Conference on Pattern Recognition. Cambridge: IEEE CS, 2004: 494497.

［10］Cawley G, Talbot N. Fast exact leaveoneout crossvalidation of sparse leastsquares support vector machines［J］. Neural Networks, 2004, 17(10): 14671475.

［11］Cawley G C, Talbot N L C. Efficient leaveoneout crossvalidation of kernel fisher discriminant classifiers［J］. Pattern Recognition, 2003, 36(11): 25852592.

[1]	Li Qingwei, Fu Can, Xue Wenli, Wei Yongqiang, Shen Zhiwen. Novel State of Health Estimation for Lithium-Ion Battery Based on Differential Evolution Algorithm-Extreme Learning Machine [J]. J Shanghai Jiaotong Univ Sci, 2025, 30(2): 252-261.
[2]	HE Zhizhuo, ZHANG Ying, ZHENG Gang, ZHENG Fang, HUANG Wandi, ZHANG Shenxi, CHENG Haozhong. Interval Prediction Technology of Photovoltaic Power Based on Parameter Optimization of Extreme Learning Machine [J]. Journal of Shanghai Jiao Tong University, 2024, 58(3): 285-294.
[3]	YANG Bo, ZENG Chunyuan, CHEN Yijun, SHU Hongchun, CAO Pulin. Extreme Learning Machine and Its Application in Parameter Identification of Proton Exchange Membrane Fuel Cell [J]. Journal of Shanghai Jiao Tong University, 2023, 57(4): 482-494.
[4]	XI Jianhui, JIANG Han, CHEN Bo, FU Li. Infrared Multispectral Radiation Temperature Measurement Based on PCA-ELM [J]. Journal of Shanghai Jiao Tong University, 2021, 55(7): 891-898.
[5]	ZHANG Linlin,HU Xiongwei,LI Peng,SHI Fang,YU Zhihong. Power System Transient Stability Assessment Based on Extreme Learning Machine [J]. Journal of Shanghai Jiaotong University, 2019, 53(6): 749-756.
[6]	ZHONG Guoqiang,WANG Hao,ZHANG Guohua,QIN Weimin WANG Chengtang,XIONG Junfeng. Analysis and Prediction of Factors Affecting Horizontal Displacement of Foundation Pit Based on RS-MIV-ELM Model [J]. Journal of Shanghai Jiao Tong University, 2018, 52(11): 1508-1515.
[7]	WU Hong,LIU Bing,ZHAO Yingxin,WANG Chong,ZHANG Yuting TANG Ran,MA Xiaoxu,XU Xiyan,MU Weiwei,LIU Zhiyang. Research on Interference Suppression in Generalized Frequency Division Multiplex System [J]. Journal of Shanghai Jiao Tong University, 2017, 51(9): 1117-1123.
[8]	ZHANG Wei，XU Aiqiang，GAO Mingzhe. An Online Condition Prediction Algorithm Based on Cumulative Coherence Measurement [J]. Journal of Shanghai Jiao Tong University, 2017, 51(11): 1391-1398.
[9]	LIU Feifei,PENG Di,HE Yanlin,ZHU Qunxiong. Research and Chemical Application of Extreme Learning Based Process Neural Network [J]. Journal of Shanghai Jiaotong University, 2014, 48(07): 977-981.
[10]	ZHANG Yingtang,MA Chao,LI Zhining,FAN Hongbo. Online Modeling of Kernel Extreme Learning Machine Based on Fast Leave-One-Out CrossValidation [J]. Journal of Shanghai Jiaotong University, 2014, 48(05): 647-652.
[11]	PAN Feng1* (潘峰), ZHAO Hai-bo2 (赵海波). Online Sequential Extreme Learning Machine Based Multilayer Perception with Output Self Feedback for Time Series Prediction [J]. Journal of shanghai Jiaotong University (Science), 2013, 18(3): 366-375.