To improve the restricted Boltzmann machine (RBM)’s data generalization ability and resolve the features homogenization problem, a random RBM group (RRBMG) design is proposed. The dimensions of observation data were randomly divided into groups, and the childRBMs were built based on the combined data group. Two methods based on the structural stories were used to compose hidden units’ layer finally, shallow structure by mean output, and deep structure through the formation of highlevel hidden units’ layer. The theoretical analysis shows that, with the increase of models’ number in the group, the training objectives of RRBMG will gradually approach the training objectives of standard RBM, and can effectively reduce the impact of feature homogeneity. The experimental results show that, compared with dropout algorithm, the proposed RRBMG can effectively improve the feature learning ability of RBM, and use the shallow structure and deep structure features to increase the classification accuracy of mixed national institute of standards and technology (MNIST) database experiment by 2% and 0.4%.
LIU Kaia，ZHANG Liminb，ZHOU Lijuna
. Design of Random Restricted Boltzmann Machine Group[J]. Journal of Shanghai Jiaotong University, 2017
, 51(10)
: 1235
-1240
.
DOI: 10.16183/j.cnki.jsjtu.2017.10.013
［1］LEE H, EKANADHAM C, NG A Y. Sparse deep belief net model for visual area V2［C］∥Proceedings of Advances in Neural Information Processing Systems. NY, United States: Curran Associates Inc, 2008: 873880.
［2］LUO H, SHEN R, NIU C. Sparse group restricted Boltzmann machines［C］∥Proceedings of 25th AAAI Conference on Artificial Intelligence and the 23rd Innovative Applications of Artificial Intelligence Conference. CA, United States: AI Access Foundation, 2011: 429434.
［3］JIN N, Zhang J S, ZHANG C X. A sparseresponse deep belief network based on rate distortion theory［J］. Pattern Recognition, 2014, 47(9): 31793191.
［4］BREULEUX O, BENGIO Y, VINCENT P. Quickly generating representative samples from an RBMderived process［J］. Neural Computation, 2011, 23(8): 20582073.
［5］BARTHELME S, CHOPIN N. The Poisson transform for unnormalised statistical models［J］. Statistics & Computing, 2014, 25(4): 114.
［6］胡洋. 基于马尔可夫链蒙特卡罗方法的RBM学习算法改进［D］. 上海: 上海交通大学计算机科学与工程系, 2012.
［7］TOSUN H, SHEPPARD J W. Training restricted Boltzmann machines with overlapping partitions［C］∥Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin: Springer, 2014:195208.
［8］罗恒. 基于协同过滤视角的受限玻尔兹曼机研究［D］. 上海: 上海交通大学 计算机科学与工程系, 2011.
［9］MIN M R, NING X, CHENG C, et al. Interpretable sparse highorder Boltzmann machines［J］. Communications IET, 2014, 10(1): 614622.
［10］MNIH V, LAROCHELLE H, HINTON G E. Conditional restricted Boltzmann machines for structured output prediction［C］∥Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence. Arlington, United States: AUAI Press, 2011: 514522.
［11］SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from overfitting［J］. The Journal of Machine Learning Research, 2014, 15(1): 19291958.
［12］刘凯, 张立民, 张超. 受限玻尔兹曼机的新混合稀疏惩罚机制［J］. 浙江大学学报(工学版), 2015, 49(6): 10701078.
LIU Kai, ZHANG Limin, ZHANG Chao. New hybrid sparse penalty mechanism of restricted Boltzmann machine［J］. Journal of Zhejiang University (Engineering Science), 2015, 49(6): 10701078.
［13］SALAKHUTDINOV R, HINTON G E. Deep Boltzmann machines［C］∥Proceedings of 12th International Conference on Artificial Intelligence and Statistics. MA, United States: Microtome Publishing Brookline, 2009: 448455.
［14］SRIVASTAVA N. Improving neural networks with dropout［D］. Toronto: University of Toronto, 2013.
［15］GOODFELLOW I J, WARDEFARLEY D, MIRZA M, et al. Maxout networks［J］. Computer Science, 2013, 28(3): 13191327.
