Online Semi-Supervised Extreme Learning Machine Based on Manifold Regularization

Abstract

Abstract:

Abstract： In this paper, with the help of the rules of block matrix multiplication, an online semi-supervised extreme learning machine(OSSELM) was proposed according to semi-supervised extreme learning machine (SS-ELM) based on manifold regularization.By the analysis of the manifoldregularization term of the objective function of SSELM, a kind of approximation algorithm of OSS-ELM named OSS-ELM(buffer) was proposed to avoid running out of memory in the process of online learning.The linear relationship between the sample number and the cumulative running time of the OSS-ELM(buffer) was revealed in the experiments using Abalone and the relative deviation of the generalization ability of the OSSELM and the SS-ELM is less than 1% in 9 public data sets, which show that the OSSELM(buffer) not only solves the problem of limited memory, but also improves the speed of online learning while keeping the generalization ability of SSELM. This proves that the OSSELM(buffer)can be effectively applied to online semisupervised learning.

Key words: extreme learning machine(ELM), semi-supervised learning, online learning, manifold regularization

CLC Number:

TP 183

WANG Ping,WANG Di,FENG Wei. Online Semi-Supervised Extreme Learning Machine Based on Manifold Regularization[J]. Journal of Shanghai Jiaotong University, 2015, 49(08): 1153-1158.

References

［1］Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: A new learning scheme of feedforward neural networks［C］∥Neural Networks, Proceedings of 2004 IEEE International Joint Conference on. Budapest：IEEE， 2004：985990.

［2］Huang G B, Zhou H, Ding X, et al. Extreme learning machine for regression and multiclass classification［J］. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on，2012，42(2)：513529.

［3］Huang G B, Wang D H, Lan Y. Extreme learning machines: a survey［J］. International Journal of Machine Learning and Cybernetics，2011，2(2)：107122.

［4］Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: theory and applications［J］. Neurocomputing，2006，70(1)：489501.

［5］Huang G B. An insight into extreme learning machines: Random neurons, random features and kernels［J］. Cognitive Computation，2014，6(3)：376390.

［6］张弦，王宏力. 局域极端学习机及其在状态在线监测中的应用［J］. 上海交通大学学报，2011，45(2)：236240.

ZHANG Xian, WANG Hongli. Local extreme learning machine and its application to condition online monitoring［J］. Journal of Shanghai Jiaotong University，2011，45(2)：236240.

［7］刘学艺,李平,郜传厚. 极限学习机的快速留一交叉验证算法［J］. 上海交通大学学报，2011，45(8)：11401145.

LIU Xueyi, LI Ping, GAO Chuanhou. Fast leaveoneout crossvalidation algorithm for extreme learning machine［J］. Journal of Shanghai Jiaotong University，2011，45(8)：11401145.

［8］张英堂，马超，李志宁，等. 基于快速留一交叉验证的核极限学习机在线建模［J］. 上海交通大学学报，2014，48(5)：641646.

ZHANG Yingtang, MA Chao, LI Zhining, et al. Online modeling of kernel extreme learning machine based on fast leaveoneout crossvalidation［J］. Journal of Shanghai Jiaotong University，2014，48(5)：641646.

［9］Huang G B, Song S J, Jatinder N D G, et al. Semisupervised and unsupervised extreme learning machines［J］. IEEE Transactions on Cybernetics, accepted paper，2014，44(12)：24052417.

［10］Chapelle O, Schlkopf B, Zien A. Semisupervised learning［M］. London: The MIT Press, 2006.

［11］Zhu X J. Semisupervised learning literature survey［R］.1530，Madison：University of Wisconsin–Madison, 2008.

［12］Andrew B G, Li M, Zhu X J. Online manifold regularization: A new learning setting and empirical study［C］∥Machine Learning and Knowledge Discovery in Databases Lecture Notes in Computer Science. Antwerp：Springer Berlin Heidelberg，2008：393407.

［13］Sun B L, Li G H, Jia L, et al. Online manifold regularization by dual ascending procedure. Mathematical Problems in Engineering［EB/OL］. 2013. http:∥www.hindawi.com/journals/ mpe/2013/838439/.

［14］黎铭，周志华. 基于多核集成的在线半监督学习方法［J］. 计算机研究与发展，2008，45(12)：20602068.

LI Ming, ZHOU Zhihua. Online SemiSupervised learning with multikernel ensemble［J］. Journal of Computer Research and Development，2008，45(12)：20602068.

[1]	JIA Dengqiang* (贾灯强), LUO Xinzhe (罗鑫喆), DING Wangbin (丁王斌),HUANG Liqin (黄立勤), ZHUANG Xiahai (庄吓海). SeRN: A Two-Stage Framework of Registration for Semi-Supervised Learning for Medical Images [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(2): 176-189.
[2]	TU Enmei,YANG Jie. A Review of Semi-Supervised Learning Theories and Recent Advances [J]. Journal of Shanghai Jiaotong University, 2018, 52(10): 1280-1291.
[3]	TANG Qi-feng (汤奇峰), LI De-wei* (李德伟), XI Yu-geng (席裕庚). Soft-Sensing Method with Online Correction Based on Semi-Supervised Learning [J]. Journal of shanghai Jiaotong University (Science), 2015, 20(2): 171-176.
[4]	MA Bo，ZHOU Yue. A New Multi-view Face Tracking Algorithm [J]. Journal of Shanghai Jiaotong University, 2010, 44(07): 902-0906.