随机受限玻尔兹曼机组设计

doi:10.16183/j.cnki.jsjtu.2017.10.013

摘要/Abstract

摘要： 为提高受限玻尔兹曼机(Restricted Boltzmann Machine,RBM)数据学习能力和抑制训练的特征同质化问题，提出一种随机受限玻尔兹曼机组(RandomRBM Group,RRBMG)设计.对观测数据进行随机维度组合，在随机维度组合的基础上构建子RBM群组并实施训练，随后依据神经网络的层数选择模型特征组合方式，针对浅层结构设置为均值组合方式，针对深层模型设置为隐单元叠加方式.理论分析表明，随着组内模型数目的增加，RRBMG所要学习的训练目标将逐渐接近于标准RBM的训练目标，并且能够有效减少特征同质化带来的影响；实验结果表明，与衰落机制相比，RRBMG能够有效提高RBM的特征学习能力，应用所组建的浅层结构和深层结构特征，将MNIST(Mixed National Institute of Standards and Technology)数据库实验的分类准确率分别提高了2%和0.4%.

关键词: , 机器学习, 深度学习, 受限玻尔兹曼机, 深度玻尔兹曼机

Abstract: To improve the restricted Boltzmann machine (RBM)’s data generalization ability and resolve the features homogenization problem, a random RBM group (RRBMG) design is proposed. The dimensions of observation data were randomly divided into groups, and the childRBMs were built based on the combined data group. Two methods based on the structural stories were used to compose hidden units’ layer finally, shallow structure by mean output, and deep structure through the formation of highlevel hidden units’ layer. The theoretical analysis shows that, with the increase of models’ number in the group, the training objectives of RRBMG will gradually approach the training objectives of standard RBM, and can effectively reduce the impact of feature homogeneity. The experimental results show that, compared with dropout algorithm, the proposed RRBMG can effectively improve the feature learning ability of RBM, and use the shallow structure and deep structure features to increase the classification accuracy of mixed national institute of standards and technology (MNIST) database experiment by 2% and 0.4%.

Key words: , machine learning, deep learning, restricted Boltzmann machine (RBM), deep Boltzmann machine

中图分类号:

TP 181

刘凯a，张立民b，周立军a. 随机受限玻尔兹曼机组设计[J]. 上海交通大学学报（自然版）, 2017, 51(10): 1235-1240.

LIU Kaia，ZHANG Liminb，ZHOU Lijuna. Design of Random Restricted Boltzmann Machine Group[J]. Journal of Shanghai Jiaotong University, 2017, 51(10): 1235-1240.

参考文献

［1］LEE H, EKANADHAM C, NG A Y. Sparse deep belief net model for visual area V2［C］∥Proceedings of Advances in Neural Information Processing Systems. NY, United States: Curran Associates Inc, 2008: 873880.
［2］LUO H, SHEN R, NIU C. Sparse group restricted Boltzmann machines［C］∥Proceedings of 25th AAAI Conference on Artificial Intelligence and the 23rd Innovative Applications of Artificial Intelligence Conference. CA, United States: AI Access Foundation, 2011: 429434.
［3］JIN N, Zhang J S, ZHANG C X. A sparseresponse deep belief network based on rate distortion theory［J］. Pattern Recognition, 2014, 47(9): 31793191.
［4］BREULEUX O, BENGIO Y, VINCENT P. Quickly generating representative samples from an RBMderived process［J］. Neural Computation, 2011, 23(8): 20582073.
［5］BARTHELME S, CHOPIN N. The Poisson transform for unnormalised statistical models［J］. Statistics & Computing, 2014, 25(4): 114.
［6］胡洋. 基于马尔可夫链蒙特卡罗方法的RBM学习算法改进［D］. 上海: 上海交通大学计算机科学与工程系, 2012.
［7］TOSUN H, SHEPPARD J W. Training restricted Boltzmann machines with overlapping partitions［C］∥Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin: Springer, 2014:195208.
［8］罗恒. 基于协同过滤视角的受限玻尔兹曼机研究［D］. 上海: 上海交通大学计算机科学与工程系, 2011.
［9］MIN M R, NING X, CHENG C, et al. Interpretable sparse highorder Boltzmann machines［J］. Communications IET, 2014, 10(1): 614622.
［10］MNIH V, LAROCHELLE H, HINTON G E. Conditional restricted Boltzmann machines for structured output prediction［C］∥Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence. Arlington, United States: AUAI Press, 2011: 514522.
［11］SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from overfitting［J］. The Journal of Machine Learning Research, 2014, 15(1): 19291958.
［12］刘凯, 张立民, 张超. 受限玻尔兹曼机的新混合稀疏惩罚机制［J］. 浙江大学学报(工学版), 2015, 49(6): 10701078.
LIU Kai, ZHANG Limin, ZHANG Chao. New hybrid sparse penalty mechanism of restricted Boltzmann machine［J］. Journal of Zhejiang University (Engineering Science), 2015, 49(6): 10701078.
［13］SALAKHUTDINOV R, HINTON G E. Deep Boltzmann machines［C］∥Proceedings of 12th International Conference on Artificial Intelligence and Statistics. MA, United States: Microtome Publishing Brookline, 2009: 448455.
［14］SRIVASTAVA N. Improving neural networks with dropout［D］. Toronto: University of Toronto, 2013.
［15］GOODFELLOW I J, WARDEFARLEY D, MIRZA M, et al. Maxout networks［J］. Computer Science, 2013, 28(3): 13191327.

[1]	曾国治, 魏子清, 岳宝, 丁云霄, 郑春元, 翟晓强. 基于CNN-RNN组合模型的办公建筑能耗预测[J]. 上海交通大学学报, 2022, 56(9): 1256-1261.
[2]	吴庶宸, 戚宗锋, 李建勋. 基于深度学习的智能全局灵敏度分析[J]. 上海交通大学学报, 2022, 56(7): 840-849.
[3]	贾岛, 陈磊, 朱志鹏, 余曜, 迟德建. 机器学习在引战系统设计中的应用研究[J]. 空天防御, 2022, 5(2): 27-31.
[4]	王聚团, 戚晓宁, 黄志明. 水下生产管汇测试技术及其改进研究[J]. 海洋工程装备与技术, 2022, 9(2): 43-49.
[5]	袁振钦, 邹科, 孙亚峰, 刘刚, 屈衍, 李居跃. 基于时域分析法的动态电缆疲劳分析[J]. 海洋工程装备与技术, 2022, 9(2): 50-55.
[6]	吕超凡, 言颖杰, 林力, 柴岗, 鲍劲松. 基于点云语义分割算法的下颌角截骨面设计[J]. 上海交通大学学报, 2022, 56(11): 1509-1517.
[7]	王卓鑫, 赵海涛, 谢月涵, 任翰韬, 袁明清, 张博明, 陈吉安. 反向传播神经网络联合遗传算法对复合材料模量的预测[J]. 上海交通大学学报, 2022, 56(10): 1341-1348.
[8]	陶海红, 闫莹菲. 一种基于GA-CNN的网络化雷达节点遴选算法[J]. 空天防御, 2022, 5(1): 1-5.
[9]	李川, 聂熠文, 刘军伟, 孟凡钦, 沈晓静. 基于机器学习的多算法融合航迹稳健起始方法[J]. 空天防御, 2022, 5(1): 20-24.
[10]	金丽洁, 武亚涛. 基于双CNN的雷达信号调制类型识别方法[J]. 空天防御, 2022, 5(1): 66-70.
[11]	王娟, 杨明旺, 郑茂尧, 刘凌云, 赵立君. 高强钢在大型半潜式平台组块建造中的应用[J]. 海洋工程装备与技术, 2022, 9(1): 27-31.
[12]	陈欣, 赵晓磊, 王立坤, 肖德明, 张腾月. 深水大型吸力锚建造技术研究[J]. 海洋工程装备与技术, 2022, 9(1): 32-36.
[13]	尹彦坤, 易涤非. 半潜式生产平台船体结构关键节点工程临界评估[J]. 海洋工程装备与技术, 2022, 9(1): 52-57.
[14]	周毅, 秦康平, 孙近文, 范栋琦, 郑义明. 台风气象环境电网设备风险量化预警及其N-m故障处置预案在线生成方法[J]. 上海交通大学学报, 2021, 55(S2): 22-30.
[15]	王兴志, 翟海保, 严亚勤, 吴庆曦. 基于数字孪生和深度学习的新一代调控系统预调度方法[J]. 上海交通大学学报, 2021, 55(S2): 37-41.