基于多头卷积残差连接的文本数据实体识别
Text data entity recognition based on muti-head convolution residual connections
刘微 1李波 1杨思瑶1
作者信息
- 1. 沈阳理工大学 信息科学与工程学院,辽宁 沈阳 110158
- 折叠
摘要
为构建工作报告中的文本数据关系型数据库,针对非结构化文本数据中有效信息实体提取问题以及传统网络在提取信息时特征丢失问题,设计了一种基于深度学习的实体识别模型RoBERTa-MCR-BiGRU-CRF,首先利用预训练模型RoBERTa作为编码器,将训练后的词向量输入到多头卷积残差网络层MCR扩充语义信息,接着输入到门控循环BiGRU层进一步提取上下文特征,最后经过条件随机场CRF层解码进行标签判别.经过实验,模型在工作报告数据集上F1值达到96.64％,优于其他对比模型;并且在数据名称实体类别上,F1值分别比BERT-BiLSTM-CRF和Ro-BERTa-BiGRU-CRF提高了3.18％、2.87％,结果表明该模型能较好地提取非结构化文本中的有效信息.
Abstract
To construct a relational database for text data in work reports,and address the problem of extracting useful information entities from unstructured text and feature loss in traditional networks during information extraction,a deep learning-based entity recognition model,which is named RoBERTa-MCR-BiGRU-CRF is proposed.The model firstly uses the pre-trained model Ro-bustly Optimized BERT Pretraining Approach(RoBERTa)as an encoder,feeding the trained word embeddings into the Multi-head Convolutional Residual network(MCR)layer to enrich semantic information.Next,the embeddings are input into a gated recurrent Bidirectional Gated Recurrent Unit(BiGRU)layer to further capture contextual features.Finally,a Conditional Ran-dom Field(CRF)layer is used for decoding and label prediction.Experimental results show that the model achieves an F1 score of 96.64％on the work report dataset,outperforming other comparative models.Additionally,for named entity categories in the data,the F1 score is 3.18％and 2.87％higher than BERT-BiLSTM-CRF and RoBERTa-BiGRU-CRF,respectively.The results demonstrate the model's effectiveness in extracting useful information from unstructured text.
关键词
深度学习/命名实体识别/神经网络/数据挖掘Key words
deep learning/named entity recognition/neural networks/data mining引用本文复制引用
出版年
2024
国家科技期刊平台
NSTL
万方数据