Named Entity Identification of Chinese Poetry and Wine Culture Based on ALBERT

doi:10.1007/s12204-023-2675-y

Abstract

Abstract: The task of identifying Chinese named entities of Chinese poetry and wine culture is a key step in the construction of a knowledge graph and a question and answer system. Aimed at the characteristics of Chinese poetry and wine culture entities with different lengths and high training cost of named entity recognition models at the present stage, this study proposes a lite BERT+bi-directional long short-term memory+ attentional mechanisms +conditional random field (ALBERT+BILSTM+Att+CRF). The method first obtains the characterlevel semantic information by ALBERT module, then extracts its high-dimensional features by BILSTM module, weights the original word vector and the learned text vector by attention layer, and finally predicts the true label in CRF module (including five types: poem title, author, time, genre, and category). Through experiments on data sets related to Chinese poetry and wine culture, the results show that the method is more effective than existing mainstream models and can efficiently extract important entity information in Chinese poetry and wine culture, which is an effective method for the identification of named entities of varying lengths of poetry.

Key words: poetry and wine culture, named entity identification, deep learning, ALBERT, bi-directional long short-term memory (BILSTM), attentional mechanisms (Att), conditional random field (CRF)

摘要： 中国诗酒文化中文命名实体识别任务是构建该领域知识图谱与问答系统的关键步骤；针对中国诗酒文化实体长短不一，以及现阶段命名实体识别模型训练成本高的特点，本研究提出一种轻量级BERT-双向长短期记忆网络-注意力机制-条件随机场(ALBERT-BILSTM-Att-CRF)的中国诗酒文化深度识别方法。该方法首先通过ALBERT模块获得字符级别的语义信息，然后由BILSTM模块抽取其高维特征，由Attention层对原始词向量和学习后的文本向量进行加权，最后在CRF模块预测出真实的标签(包括：诗词题目，作者，时间，体裁和类型五类)序列。通过对中国诗酒文化相关数据集进行实验，结果表明：该方法的效果高于现有的主流模型，可以高效提取中国诗酒文化中的重要实体信息，是一种针对长短不一诗歌类命名实体识别的有效方法。

关键词: 诗酒文化，命名实体识别，深度学习，ALBERT模型，双向长短期记忆网络(BILSTM)，注意力机制(Att)，条件随机场(CRF)

CLC Number:

TP391.1

YANG Zhuang, LI Zhaofei, WANG Jihua, WEI Xudong, ZHANG Yijie. Named Entity Identification of Chinese Poetry and Wine Culture Based on ALBERT[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(5): 1065-1072.

References

[1] LI J, SUN A X, HAN J L, et al. A survey on deep learning for named entity recognition [J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(1): 50-70.

[2] LIN Q K, MAO R, LIU J, et al. Fusing topology contexts and logical rules in language models for knowledge graph completion [J]. Information Fusion, 2023, 90: 253-264.

[3] CHEN H, HONG P F, HAN W, et al. Dialogue relation extraction with document-level heterogeneous graph attention networks [J]. Cognitive Computation, 2023, 15(2): 793-802.

[4] ZHANG D Q, YAO Y Z, LI J, et al. Synchronous condenser-based intelligent question answering system based on knowledge graph[M]//Proceedings of the world conference on intelligent and 3-D technologies (WCI3DT 2022). Singapore: Springer, 2023: 521-529.

[5] AONE C, HALVERSON L, HAMPTON T, et al. SRA: Description of the IE2 system used for MUC-7[C]//Seventh Message Understanding Conference. Fairfax: SRA: 1998.

[6] ETZIONI O, CAFARELLA M, DOWNEY D, et al. Unsupervised named-entity extraction from the Web: An experimental study [J]. Artificial Intelligence, 2005, 165(1): 91-134.

[7] HAN X, KWOH C K, KIM J J. Clustering based active learning for biomedical Named Entity Recognition [C]//2016 International Joint Conference on Neural Networks. Vancouver: IEEE, 2016: 1253-1260.

[8] WANG Z R, DU J, WANG J M. Writer-aware CNN for parsimonious HMM-based offline handwritten Chinese text recognition [J]. Pattern Recognition, 2020, 100: 107102.

[9] ALOKAILI A, EL BACHIR MENAI M. SVM ensembles for named entity disambiguation [J]. Computing, 2020, 102(4): 1051-1076.

[10] LIU S, HE T H, DAI J H. A survey of CRF algorithm based knowledge extraction of elementary mathematics in Chinese [J]. Mobile Networks and Applications, 2021, 26(5): 1891-1903.

[11] HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging [DB/OL]. (2015-08-09). https://arxiv.org/abs/1508.01991

[12] STRUBELL E, VERGA P, BELANGER D, et al. Fast and accurate entity recognition with iterated dilated convolutions [DB/OL]. (2017-02-07). https://arxiv.org/abs/1702.02098

[13] ULČAR M, ROBNIK-ŠIKONJA M. Cross-lingual alignments of ELMo contextual embeddings [J]. Neural Computing and Applications, 2022, 34(15): 13043-13061.

[14] SAVELKA J, AGARWAL A, BOGART C, et al. Large language models (GPT) struggle to answer multiple-choice questions about code [DB/OL]. (2023-03-09). https://arxiv.org/abs/2303.08033

[15] KUMAR P, RAMAN B. A BERT based dual-channel explainable text emotion recognition system [J]. Neural Networks, 2022, 150: 392-407.

[16] LAN Z Z, CHEN M D, GOODMAN S, et al. ALBERT: A lite BERT for self-supervised learning of language representations [DB/OL]. (2019-09-26). https://arxiv.org/abs/1909.11942

[17] AN Y, XIA X Y, CHEN X L, et al. Chinese clinical named entity recognition via multi-head self-attention based BiLSTM-CRF [J]. Artificial Intelligence in Medicine, 2022, 127: 102282.