外部注意力增强语义交互的阅读理解模型
Reading comprehension model based on external attention-enhancing semantic interactions
吴迪 1马超 2段晓旋1
作者信息
- 1. 河北工程大学信息与电气工程学院,河北邯郸 056038
- 2. 河北工程大学信息与电气工程学院,河北邯郸 056038;中共邢台市委 中共邢台市委办公室,河北 邢台 054001
- 折叠
摘要
针对传统抽取式阅读理解模型未充分考虑问答样本之间潜在相关性的问题,通过RoBERTa对问题与段落进行编码,利用外部注意力Exatt增强语义交互层特征获取能力,提出外部注意力增强语义交互的阅读理解模型,捕获问题与段落中蕴涵的语义特征和不同问答样本之间的潜在相关性.实验结果表明,在CMRC2018和构建的电力安规问答数据集上,在评价指标EM和F1两方面,该方法较基线模型分别最高提高了 0.737％和2.556％.
Abstract
Aiming at the problem that the traditional extractive reading comprehension model does not fully consider the potential correlation between Q&A samples,the RoBERTa was used to encode the questions and paragraphs.The external attention Exatt was used to enhance the feature acquisition ability of the semantic interaction layer.The reading comprehension model based on external attention enhances semantic interaction was proposed.The semantic features in the questions and paragraphs,the potential correlation between different Q&A samples were obtained.Experimental results show that Roberta-Exatt model can improve the evaluation indexes of EM and F1 by 0.737％and 2.556％on the constructed power safety Q&A dataset,respectively,compared with the baseline model.
关键词
电力安规/抽取式机器阅读理解/预训练模型/问答样本/潜在相关性/外部注意力/语义交互Key words
power safety/abstract machine reading comprehension/pre-training model/Q&A samples/potential relevance/ex-ternal attention/semantic interaction引用本文复制引用
基金项目
国网电网有限公司科技指南基金项目(5600-202019167A-0-0-00)
河北省自然科学基金项目(F2020402003)
出版年
2024
NETL
NSTL
万方数据