基于连续提示注入与指针网络的农业病害命名实体识别

扫码查看

原文链接

国家科技期刊平台
NETL
NSTL
万方数据

中文摘要：针对农业病害领域命名实体识别过程中存在的预训练语言模型利用不充分、外部知识注入利用率低、嵌套命名实体识别率低的问题,本文提出基于连续提示注入和指针网络的命名实体识别模型CP-MRC(Continuous prompts for machine reading comprehension).该模型引入 BERT(Bidirectional encoder representation from transformers)预训练模型,通过冻结BERT模型原有参数,保留其在预训练阶段获取到的文本表征能力;为了增强模型对领域数据的适用性,在每层Transformer中插入连续可训练提示向量;为提高嵌套命名实体识别的准确性,采用指针网络抽取实体序列.在自建农业病害数据集上开展了对比实验,该数据集包含2 933条文本语料,8个实体类型,共10 414个实体.实验结果显示,CP-MRC模型的精确率、召回率、F1值达到83.55％、81.4％、82.4％,优于其他模型;在病原、作物两类嵌套实体的识别率较其他模型F1值提升3个百分点和13个百分点,嵌套实体识别率明显提升.本文提出的模型仅采用少量可训练参数仍然具备良好识别性能,为较大规模预训练模型在信息抽取任务上的应用提供了思路.

外文标题：Named Entity Recognition of Agricultural Disease Based on Continuous Prompts Injection and Pointer Network

外文摘要：In response to the problems of insufficient utilization of pretrained language models,low utilization of external knowledge injection,and low recognition rate of nested named entities in the process of named entity recognition in the field of agricultural diseases,a named entity recognition model continuous prompts for machine reading comprehension(CP-MRC)was proposed based on continuous prompt injection and pointer network.This model introduced the bidirectional encoder representation from transformers(BERT)pretraining model,which freezed the original parameters of the BERT model and retained its text representation ability obtained during the pretraining stage.To enhance the applicability of the model to domain data,continuous trainable hint vectors were inserted into each layer of Transformer.To improve the accuracy of nested named entity recognition,a pointer network was used to extract entity sequences.A comparative experiment was conducted on a self built agricultural disease dataset,which included 2 933 text corpora,8 entity types,and a total of 10 414 entities.The experimental results showed that the accuracy,recall,and F1 values of the CP-MRC model reached 83.55％,81.4％,and 82.4％,which was superior to other models.The recognition rate of nested entities in pathogens and crops was increased by 3 percentage points and 13 percentage points in F1 value compared with that of others,and the recognition rate of nested entities was significantly improved.The model still had good recognition performance with only a small number of trainable parameters,providing ideas for the application of large-scale pretrained models in information extraction tasks.

外文关键词：

agricultural diseasesnamed entity recognitioncontinuous promptspointer networknested entitiespretrained language model

作者：

王春山、张宸硕、吴华瑞、朱华吉、缪祎晟、张立杰

展开 >

作者单位：

河北农业大学信息科学与技术学院,保定 071001

国家农业信息化工程技术研究中心,北京 100097

农业农村部农业信息化技术重点实验室,北京 100097

河北农业大学机电工程学院,保定 071001

河北省农业大数据重点实验室,保定 071001

展开 >

关键词：

农业病害命名实体识别连续提示指针网络嵌套实体预训练语言模型

基金：

国家科技创新2030-"新一代人工智能"重大项目财政部和农业农村部:国家现代农业产业技术体系项目河北省自然科学基金项目

项目编号：

2021ZD0113604CARS-23-D07F2022204004

出版年：

2024

DOI：

10.6041/j.issn.1000-1298.2024.06.027

农业机械学报

中国农业机械学会中国农业机械化科学研究院

农业机械学报

CSTPCD北大核心

影响因子：1.904

ISSN：1000-1298

年,卷(期)：2024.55(6)