融入BERT的企业年报命名实体识别方法

doi:10.16183/j.cnki.jsjtu.2020.009

上海交通大学学报 ›› 2021, Vol. 55 ›› Issue (2): 117-123.doi: 10.16183/j.cnki.jsjtu.2020.009

所属专题：《上海交通大学学报》2021年12期专题汇总专辑；《上海交通大学学报》2021年“自动化技术、计算机技术”专题

融入BERT的企业年报命名实体识别方法

张靖宜¹, 贺光辉¹(), 代洲², 刘亚东¹

^1.上海交通大学　电子信息与电气工程学院，上海　200240
^2.南方电网物资有限公司，广州　510641

收稿日期:2020-01-08 出版日期:2021-02-01 发布日期:2021-03-03
通讯作者: 贺光辉 E-mail:guanghui.he@sjtu.edu.cn
作者简介:张靖宜(1996-)，女，河南省南阳市人，硕士生，主要从事自然语言处理的研究．

Named Entity Recognition of Enterprise Annual Report Integrated with BERT

ZHANG Jingyi¹, HE Guanghui¹(), DAI Zhou², LIU Yadong¹

^1.School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
^2.China Southern Power Grid Materials Co. , Ltd. , Guangzhou 510641, China

Received:2020-01-08 Online:2021-02-01 Published:2021-03-03
Contact: HE Guanghui E-mail:guanghui.he@sjtu.edu.cn

摘要/Abstract

摘要：

自动提取企业年报关键数据是企业评价工作自动化的重要手段．针对企业年报领域关键实体结构复杂、与上下文语义关联强、规模较小的特点，提出基于转换器的双向编码器表示-双向门控循环单元-注意力机制-条件随机场(BERT-BiGRU-Attention-CRF)模型．在BiGRU-CRF模型的基础上，首先引入BERT预训练语言模型，以增强词向量模型的泛化能力，捕捉长距离的上下文信息；然后引入注意力机制，以充分挖掘文本的全局和局部特征．在自行构建的企业年报语料库内进行实验，将该模型与多组传统模型进行对比．结果表明：该模型的F₁值(精确率和召回率的调和平均数)为93.69%，对企业年报命名实体识别性能优于其他传统模型，有望成为企业评价工作自动化的有效方法．

关键词: 命名实体识别, 企业年报, BERT, 注意力机制, 双向门控循环单元

Abstract:

Automatically extracting key data from annual reports is an important means of business assessments. Aimed at the characteristics of complex entities, strong contextual semantics, and small scale of key entities in the field of corporate annual reports, a BERT-BiGRU-Attention-CRF model was proposed to automatically identify and extract entities in the annual reports of enterprises. Based on the BiGRU-CRF model, the BERT pre-trained language model was used to enhance the generalization ability of the word vector model to capture long-range contextual information. Furthermore, the attention mechanism was used to fully mine the global and local features of the text. The experiment was performed on a self-constructed corporate annual report corpus, and the model was compared with multiple sets of models. The results show that the value of F₁ (harmonic mean of precision and recall) of the BERT-BiGRU-Attention-CRF model is 93.69%. The model has a better performance than other traditional models in annual reports, and is expected to provide an automatic means for enterprise assessments.

Key words: named entity recognition, enterprise annual report, BERT, attention mechanism, BiGRU

中图分类号:

TP391.1

张靖宜, 贺光辉, 代洲, 刘亚东. 融入BERT的企业年报命名实体识别方法[J]. 上海交通大学学报, 2021, 55(2): 117-123.

ZHANG Jingyi, HE Guanghui, DAI Zhou, LIU Yadong. Named Entity Recognition of Enterprise Annual Report Integrated with BERT[J]. Journal of Shanghai Jiao Tong University, 2021, 55(2): 117-123.

图/表 10

表1

表2

图1

图2

图3

表3

表4

表5

表6

图4

参考文献 15

[1]	ZHENG S C, WANG F, BAO H Y, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. (Volume 1: Long Papers). Stroudsburg, PA, USA: ACL, 2017: 1227-1236.
[2]	吴文涛，李培峰，朱巧明．基于混合神经网络的实体和事件联合抽取方法[J]. 中文信息学报，2019, 33(8): 77-83.
	WU Wentao, LI Peifeng, ZHU Qiaoming. Joint extraction of entities and events by a hybrid neural network[J]. Journal of Chinese Information Processing, 2019, 33(8): 77-83.
[3]	HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[OL]. (2015-08-09)[2019-11-20]. .
[4]	CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[J]. Transactions of the Association for Computational Linguistics, 2016, 4(1): 357-370.
[5]	MA X Z, HOVY E. End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF[C]∥Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. (Volume 1: Long Papers). Stroudsburg, PA, USA: ACL, 2016: 1064-1074.
[6]	CHO K, VAN M B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]∥Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA, USA: ACL, 2014: 1724-1734.
[7]	王洁，张瑞东，吴晨生．基于GRU的命名实体识别方法[J]. 计算机系统应用，2018, 27(9): 18-24.
	WANG Jie, ZHANG Ruidong, WU Chensheng. Named entity recognition method based on GRU[J]. Computer Systems & Applications, 2018, 27(9): 18-24.
[8]	BHARADWAJ A, MORTENSEN D, DYER C, et al. Phonologically aware neural model for named entity recognition in low resource transfer settings[C]∥Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: ACL, 2016: 1462-1472.
[9]	CAO P F, CHEN Y B, LIU K, et al. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism[C]∥Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: ACL, 2018: 182-192.
[10]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]∥Proceedings of the 31st Conference on Neural Information Processing Systems. Long Beach, CA, USA: NIPS, 2017: 5998-6008.
[11]	DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[OL]. (2019-05-24)[2019-11-25]. .
[12]	WITTEN I H, PAYNTER G W, FRANK E, et al. KEA: Practical automatic keyphrase extraction[C]∥Proceedings of the Fourth ACM Conference on Digital Libraries. New York, USA: ACM, 1999: 254-255.
[13]	KANDOLA E J, HOFMANN T, POGGIO T, et al. A neural probabilistic language model[J]. Studies in Fuzziness and Soft Computing, 2006, 194: 137-186.
[14]	KIROS R, ZHU Y K, SALAKHUTDINOV R R, et al. Skip-thought vectors[C]∥Proceedings of the 29th Conference on Neural Information Processing Systems. Montreal, Canada: NIPS, 2015: 3294-3302.
[15]	STRUBELL E, VERGA P, BELANGER D, et al. Fast and accurate entity recognition with iterated dilated convolutions[C]∥Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: ACL, 2017: 2670-2680.

实体类别	实体示例	实体符号
年份	2005年、2016年	YEAR
企业名称	平安银行股份有限公司	COM
财务术语	净利润、营业收入、主营业务收入	FIN
利润类数值	12 626.23万元	PNUM
收入类数值	138.64亿元	SNUM
利润类数值同上年变化趋势	增长13.27%、扭亏为盈	PTREND
收入类数值同上年变化趋势	下降23.12%、创下新高	STREND

语料字符	标注符号	语料字符	标注符号
实	O	入	I-FIN
现	O	1	B-SNUM
营	B-FIN	7	I-SNUM
业	I-FIN	万	I-SNUM
收	I-FIN	元	I-SNUM

项目	字符个数	实体标记字符个数
训练集	384 192	169 263
开发集	128 064	55 294
测试集	133 006	57 426

实体符号	训练集	开发集	测试集
YEAR	4 884	1 628	1 644
COM	4 182	1 194	1 292
FIN	12 213	4 271	4 297
PNUM	3 421	1 041	1 031
SNUM	3 162	1 016	1 057
PTREND	2 031	629	604
STREND	2 154	627	649

实体符号	P	R	F₁
YEAR	98.66	98.66	98.66
COM	96.59	93.41	94.97
FIN	95.93	93.07	94.48
PNUM	92.97	89.81	91.37
SNUM	95.83	92.00	93.88
PTREND	91.46	79.44	85.03
STREND	93.05	86.73	89.73

融入BERT的企业年报命名实体识别方法

Named Entity Recognition of Enterprise Annual Report Integrated with BERT

RichHTML

PDF (PC)

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 15

相关文章 15

编辑推荐

Metrics

本文评价

模型	P	R	F₁
CRF	74.52	71.20	72.82
BiGRU-CRF	86.69	83.61	85.12
BiGRU-Attention-CRF	88.75	86.28	87.50
BERT-BiGRU-CRF	94.19	90.07	92.08
BERT-BiGRU-Attention-CRF	95.45	91.99	93.69

[1]	. 用于水下图像增强的全局密集双分支级联网络[J]. J Shanghai Jiaotong Univ Sci, 2026, 31(2): 458-474.
[2]	. 改进DeepLabv3+的高分辨率遥感图像分割方法[J]. J Shanghai Jiaotong Univ Sci, 2026, 31(2): 348-358.
[3]	. EC-BERT: 面向中文普通话语音识别BERT纠错语言模型[J]. J Shanghai Jiaotong Univ Sci, 2026, 31(2): 282-288.
[4]	刘轩, 兰晓宸, 徐大鹏, 何良, 刘茂珅. 基于深度学习时序特征增强的海杂波抑制方法[J]. 空天防御, 2026, 9(1): 36-45.
[5]	. 基于ALBERT的中国诗酒文化命名实体识别[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(5): 1065-1072.
[6]	. CenterLineFormer：基于单车载相机的车道中心线生成方法[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(5): 1009-1017.
[7]	. 基于多注意力机制的轻量化人体姿态估计[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(5): 899-910.
[8]	. 基于多特征提取方法的多场景烟雾检测[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(5): 866-879.
[9]	. 迁移学习和注意机制融合用于CT图像COVID-19病灶分割的计算机辅助诊断[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(3): 566-581.
[10]	王可, 刘奕阳, 杨杰, 鲁爱国, 李哲, 徐明亮. 基于自适应特征增强和融合的舰载机着舰拉制状态识别[J]. 上海交通大学学报, 2025, 59(2): 274-282.
[11]	汪维, 阮雅端, 顾鹏, 陈启美. 基于融合注意力机制DAUNet的最大裂缝宽度计算[J]. 上海交通大学学报, 2025, 59(12): 1866-1877.
[12]	李楚晨, 唐善军, 赵冰青. 一种基于无人机探测图像区块信息的弱小目标检测算法[J]. 空天防御, 2025, 8(1): 41-47.
[13]	丁黎辉1, 2, 付立军1, 3, 杨光4, 5, 6, 万林4, 5, 常志军7. 基于视频的婴儿癫痫性痉挛综合征检测：建模、检测与评估[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(1): 1-9.
[14]	徐旺旺1,2，许良凤1,2，刘宁徽3，律娜3. 基于多注意力卷积神经网络的乳腺癌组织学图像诊断[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(1): 91-106.
[15]	周成, 蒋祖华. 融入优质主题和注意力机制的设计规范命名实体识别方法[J]. J Shanghai Jiaotong Univ Sci, 2024, 29(6): 1169-1180.