Variational Bayesian Learning for Parameter Estimation of Mixture of Gaussians

Abstract

Abstract:

Non-Gaussian signals or systems are usually modeled by mixture of Gaussians (MoG) models containing hidden variables. A variational Bayesian learning algorithm was suggested to infer the parameters of MoG. The algorithm estimateed the parameters of MoG by iteratively maximizing the lower bound of the marginal likelihood and updating the variational parameters until the free-form distribution was sufficiently close to the true posterior. The detailed learning of variational Bayes for MoG was derived and explained. The experiments show that this method can estimate the parameters of MoG favorably with sampling method from the engineering view.

Key words: parameters estimation, mixture model, Gaussian distribution, variational Bayes

CLC Number:

TN 911.23

XU Dingjie,SHEN Chen,SHEN Feng. Variational Bayesian Learning for Parameter Estimation of Mixture of Gaussians[J]. Journal of Shanghai Jiaotong University, 2013, 47(07): 1119-1125.

References

［1］Nasri A, Schober R. Performance of BICMSC and BICMOFDM systems with diversity reception in nonGaussian noise and interference ［J］. IEEE Transactions on Communications, 2009, 57(11): 33163327.

［2］沈锋, 孙枫. 弱相关非高斯环境下基于局部最佳检测器的伪码捕获方法［J］.电子与信息学报, 2010, 32(4): 811815.

SHEN Feng, SUN Feng. PN code acquisition based on the locally optimum detector in weakly dependent nonGaussian impulsive channels ［J］. Journal of Electronics & Information Technology, 2010, 32(4): 811815.

［3］He J, Liu Z. Underwater acoustic azimuth and elevation angle estimation using spatial invariance of two identically oriented vector hydrophones at unknown locations in impulsive noise ［J］. Digital Signal Processing, 2009, 19(3): 452462.

［4］沈锋，孙枫，薛冰.一种弱相关非高斯信道下的伪码捕获方法［J］. 系统工程与电子技术, 2009, 31(7): 15681572.

SHEN Feng, SUN Feng, XUE Bing. Pseudo noisecode acquisition method in weakly dependent nonGaussian impulsive channels ［J］. Systems Engineering and Electronics, 2009, 31(7): 15681572.

［5］王平波, 蔡志明, 刘旺锁. 混合高斯参数估计的动态簇算法［J］. 声学技术, 2007, 26(4): 741746.

WANG Pingbo, CAI Zhiming, LIU Wangsuo. Dynamic cluster algorithm for Gaussian mixture parameter estimation ［J］. Technical Acoustias, 2007, 26(4): 741746.

［6］王永忠, 梁彦, 潘泉. 基于自适应混合高斯模型的时空背景建模［J］. 自动化学报, 2009, 35(4): 371378.

WANG Yongzhong, LIANG Yan, PAN Quan. Spatiotemporal background modeling based on adaptive mixture of Gaussians ［J］. Acta Automatica Sinica, 2009, 35(4): 371378.

［7］Bouguila, N. Bayesian hybrid generative discriminative learning based on finite Liouville mixture models ［J］. Pattern Recognition, 2011, 44(6): 11831200.

［8］Shen Y, Cornford D. Variational markov chain monte carlo for bayesian smoothing of nonlinear diffusions ［J］. Computational Statistics, 2012, 27(1): 149176.

［9］Emtiyaz K M. An expectationmaximization algorithm for learning the latent Gaussian model with Gaussian likelihood ［EB/OL］. (20110422) ［20120613］. http://www.cs.ubc.ca/~emtiyaz/Writings/FA1.pdf.

［10］朱慧明，韩玉启. 贝叶斯多元统计推断理论［M］. 北京：科学出版社. 2006: 25.

［11］Attias H. A variational Bayesian framework for graphical models ［C］∥Advances in Neural Information Processing Systems 12. Cambridge, MA:［s.n.］, 2000: 209215.

［12］Vrettas M D, Cornford D, Opper M. Estimating parameters in stochastic systems: a variational Bayesian approach ［J］. Physica D, 2011, 240(23): 18771900.

［13］Sun S J, Peng C L, Hou W S. Blind source separation with time series variational Bayes expectation maximization algorithm ［J］. Digital Signal Processing, 2012, 22(1): 1733.

［14］Huang Q H, Yang J, Zhou Y. Variational Bayesian method for speech enhancement ［J］. Neurocomputing, 2007, 70(1618): 30633067.

［15］Armagan A, Zaretzki R L. A note on meanfield variational approximations in Bayesian probit models ［J］. Computational Statistics & Data Analysis, 2011, 55(1): 641643.

［16］Ishikawa Y, Takeuchi I, Nakanob R. Multidirectional search from the primitive initial point for Gaussian mixture estimation using variational Bayes method ［J］. Neural Networks, 2010, 23(3): 356364.

［17］Beal M J. Variational algorithms for approximate bayesian inference ［D］. London: University College London, 2003.

[1]	LIU Lin, YANG Siyu, HUANG Xianan, CHEN Yantao, XU Huashuai, WANG Lingling, JIANG Chuanwen. Data-Model Hybrid-Driven Simulation Method for Electric Vehicle Fleet Behavior [J]. Journal of Shanghai Jiao Tong University, 2026, 60(1): 51-60.
[2]	HU Mingxuan, QIAO Jun, ZHANG Zhinan. Segmentation and Evaluation of Continuous Rehabilitation Exercises [J]. Journal of Shanghai Jiao Tong University, 2023, 57(5): 533-544.
[3]	ZHANG Wenan, GAO Weizhan, LIU Andong. Robot Skill Learning Based on Dynamic Motion Primitives and Adaptive Control [J]. Journal of Shanghai Jiao Tong University, 2023, 57(3): 354-365.
[4]	ZHENG Dezhong, YANG Yuanyuan, XIE Zhe, NI Yangfan, LI Wentao. Data Splitting Method of Distance Metric Learning Based on Gaussian Mixed Model [J]. Journal of Shanghai Jiao Tong University, 2021, 55(2): 131-140.
[5]	DONG Xiangxiang, LÜ Runyan, CAI Yunze*. A Variational Bayes-Based Filter with Uncertain Heavy-Tailed Noise [J]. Journal of Shanghai Jiaotong University, 2020, 54(9): 881-889.
[6]	LAI Junfeng (赖俊峰), JI Dandan (季丹丹), YAN Zaizai (闫在在). Extended Inverse Gaussian Distribution: Properties and Application [J]. J Shanghai Jiaotong Univ Sci, 2020, 25(2): 193-200.
[7]	ZHANG Zhuqing (张铸青), DONG Peng (董鹏), TUO Hongya (庹红娅), LIU Guangjun (刘光军), JIA He (贾鹤). Robust Variational Bayesian Adaptive Cubature Kalman Filtering Algorithm for Simultaneous Localization and Mapping with Heavy-Tailed Noise [J]. Journal of Shanghai Jiao Tong University (Science), 2020, 25(1): 76-87.
[8]	JIA Xiaoli,JIANG Xiaobo,JIANG Sanxin，LIU Peilin. A Reconstruction Algorithm for Speech Compressive Sensing Using Structural Features [J]. Journal of Shanghai Jiao Tong University, 2017, 51(9): 1111-1116.
[9]	YU Haoa，SUN Tanfenga,b，JIANG Xinghaoa,b. Video Anomaly Detection Based on Statistic Feature of Optical Flow Block [J]. Journal of Shanghai Jiaotong University, 2015, 49(08): 1199-1204.
[10]	CHEN Qinghuaa,b,XU Manmana,PANG Lia,LIU Zegongb,c,GUAN Weijuand. Unsteady Inverse heat Conduction Problem of Loose Coal Bulk in First Kind Boundary Condition [J]. Journal of Shanghai Jiaotong University, 2014, 48(12): 1809-1814.
[11]	LEI Ju- Yang. Speech Enhancement Based on Chinese Restaurant Process [J]. Journal of Shanghai Jiaotong University, 2013, 47(04): 635-639.
[12]	LV　Fu-Qiang-1 , WANG Hui-1, LIU Hong-2. Traffic Flow Background Detection Algorithm Based on Improved Gaussian Mixture Model [J]. Journal of Shanghai Jiaotong University, 2012, 46(12): 1920-1925.
[13]	JIN Yong-Xing-a, ZHANG Shi-Bin-b. Statistical Analysis on Short-Term Distributions of Wave-Induced Stresses for Container Vessel’s Hull [J]. Journal of Shanghai Jiaotong University, 2012, 46(08): 1243-1247.