A Heterogeneous Network Representation Method Based on Variational Inference and Meta-Path Decomposition

doi:10.16183/j.cnki.jsjtu.2020.187

Abstract

Abstract:

Aimed at the problem that the traditional meta-path random walk in heterogeneous network representation cannot accurately describe the heterogeneous network structure and cannot capture the true distribution of network nodes well, a heterogeneous network representation method based on variational inference and meta-path decomposition is proposed, which is named HetVAE. First, combining with the idea of path similarity, a node selection strategy is designed to improve the random walk of the meta-path. Next, the variational theory is introduced to effectively sample the latent variables in the original distribution. After that, a personalized attention machanism is implemented, which weights the node vector representation of different sub-networks obtained by decomposition. Then, these node vectors are fused by the proposed model, so that the final node vector representation can have richer semantic information. Finally, several experiments on different network tasks are performed on the three real data sets of DBLP, AMiner, and Yelp. The effectiveness of the model is verified by these results. In node classification and node clustering tasks, compared with some state-of-the-art algorithms, the Micro-F₁ and normalized mutual information (NMI) increase by 1.12% to 4.36% and 1.35% to 18% respectively. It is proved that HetVAE can effectively capture the heterogeneous network structure and learn the node vetcor representation that conforms more with the true distribution.

Key words: heterogeneous network, network representation, variational autoencoder, random walk, attention mechanism

CLC Number:

TP181

YUAN Ming, LIU Qun, SUN Haichao, TAN Hongsheng. A Heterogeneous Network Representation Method Based on Variational Inference and Meta-Path Decomposition[J]. Journal of Shanghai Jiao Tong University, 2021, 55(5): 586-597.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Fig.4

Tab.1

Fig.5

Tab.2

Fig.6

Fig.7

References 22

[1]	CUI P, WANG X, PEI J, et al. A survey on network embedding[EB/OL]. (2017-11-23)[2019-12-22]. https://arxiv.org/abs/1711.08752
[2]	涂存超, 杨成, 刘知远, 等. 网络表示学习综述[J]. 中国科学: 信息科学, 2017, 47(8):980-996.
	TU Cunchao, YANG Cheng, LIU Zhiyuan, et al. Network representation learning: An overview[J]. Scientia Sinica (Informationis), 2017, 47(8):980-996.
[3]	PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: Online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining-KDD' 14 New York, NY, USA: ACM Press, 2014: 701-710.
[4]	GROVER A, LESKOVEC J. Node2vec: Scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY, USA: ACM Press, 2016: 855-864.
[5]	ZHU D Y, CUI P, WANG D X, et al. Deep variational network embedding in Wasserstein space[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY,USA: ACM Press, 2018: 2827-2836.
[6]	TANG J, QU M, WANG M Z, et al. LINE: Large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web-WWW '15 New York, NY, USA: ACM Press, 2015: 1067-1077.
[7]	FU T Y, LEE W C, LEI Z. HIN2Vec: Explore meta-paths in heterogeneous information networks for representation learning[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management New York, NY, USA: ACM Press, 2017: 1797-1806.
[8]	DONG Y X, CHAWLA N V, SWAMI A. Metapath2vec: Scalable representation learning for heterogeneous networks[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY, USA: SACM Press, 2017: 135-144.
[9]	TANG J, QU M, MEI Q Z. PTE: Predictive text embedding through large-scale heterogeneous text networks[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD '15. New York, NY, USA: ACM Press, 2015: 1165-1174.
[10]	XU L C, WEI X K, CAO J N, et al. Embedding of embedding (EOE): Joint embedding for coupled heterogeneous networks[C]//Proceedings of the Tenth ACM International Conference on Web Search and Data Mining-WSDM '17. New York, NY, USA: ACM Press, 2017: 741-749.
[11]	CHANG S Y, HAN W, TANG J L, et al. Heterogeneous network embedding via deep architectures[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD '15. New York, NY, USA: ACM Press, 2015: 119-128.
[12]	WANG H W, ZHANG F Z, HOU M, et al. SHINE: Signed heterogeneous information network embedding for sentiment link prediction[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining-WSDM '18 New York, NY, USA: ACM Press, 2018: 592-600.
[13]	QU M, TANG J, HAN J W. Curriculum learning for heterogeneous star network embedding via deep reinforcement learning[C]//Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining-WSDM '18. New York, NY, USA: ACM Press, 2018: 468-476.
[14]	WANG X, JI H Y, SHI C, et al. Heterogeneous graph attention network[C]//The World Wide Web Conference. New York, NY, USA: ACM Press, 2019: 2022-2032.
[15]	ZHANG C X, SONG D J, HUANG C, et al. Heterogeneous graph neural network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY, USA: ACM Press, 2019: 793-803.
[16]	CEN Y K, ZOU X, ZHANG J W, et al. Representation learning for attributed multiplex heterogeneous network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY, USA: ACM Press, 2019: 1358-1368.
[17]	HU B B, ZHANG Z Q, SHI C, et al. Cash-out user detection based on attributed heterogeneous information network with a hierarchical attention mechanism[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33:946-953. doi: 10.1609/aaai.v33i01.3301946 URL
[18]	SHI C, LI Y T, ZHANG J W, et al. A survey of heterogeneous information network analysis[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(1):17-37. doi: 10.1109/TKDE.2016.2598561 URL
[19]	SUN Y, HAN J, YAN X, et al. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks[J]. Proceedings of the VLDB Endowment, 2011, 4(11):992-1003. doi: 10.14778/3402707.3402736 URL
[20]	KINGMA D P, WELLING M. Auto-encoding variational bayes[EB/OL].(2014-05-01) [2019-12-22]. https://arxiv.org/abs/1312.6114 .
[21]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. (2014-05-01) [2019-12-22]. https://arxiv.org/abs/1706.03762 .
[22]	SHI C, HU B B, ZHAO W X, et al. Heterogeneous information network embedding for recommendation[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(2):357-370. doi: 10.1109/TKDE.2018.2833443 URL

数据集	链边关系 (A-B)	A类型节点的数量	B类型节点的数量	A和B链边关系的数量	标签数量	标签类别	元路径	A类型节点平均度	B类型节点平均度	网络平均度
DBLP	Pa-A	14376	14475	41794	4057	4	APaA	11.88	2.89	4.73
	Pa-C	14376	20	14376			APaCPaA		718.8
	Pa-T	14376	8920	114624			APaTPaA		12.85
AMiner	Pa-A	13978	16543	52957	9726	8	APaA	4.79	3.20	2.05
	Pa-C	13978	2152	13978			APaCPaA		6.49
Yelp	Bu-S	2614	2	2614	2614	3	BuSBu	13.79	1370.0	9.22
	Bu-St	2614	8	2614			BuStBu		326.75
	Bu-U	2614	1286	30838			BuUBu		23.98

算法	DBLP		AMiner		Yelp
算法	NMI	ARI	NMI	ARI	NMI	ARI
Deepwalk	0.5841	0.4960	0.3160	0.2227	0.2940	0.3179
Node2vec	0.5401	0.4776	0.3081	0.2219	0.0105	0.0111
HIN2vec	0.0124	0.0106	0.1670	0.0758	0.1353	0.1708
Metapath2vec	0.6395	0.6369	0.2645	0.2083	0.3540	0.4047
HERec	0.6844	0.7104	0.3230	0.2322	0.3511	0.4018
HAN	0.5987	0.5929	0.0375	0.0165	0.3635	0.4255
HetVAE_rw	0.7742	0.8329	0.3324	0.2234	0.3603	0.4016
HetVAE_sk	0.8173	0.8664	0.3446	0.2321	0.3593	0.4097
HetVAE_con	0.7826	0.8351	0.3239	0.2660	0.3416	0.3917
HetVAE	0.8540	0.9016	0.4025	0.3798	0.3761	0.4399