基于KS-BERT算法对短文本匹配方法的研究
A Study on Short Text Matching Method Based on KS-BERT Algorithm
杨浩文 1孙美凤2
作者信息
- 1. 扬州大学 信息工程(人工智能)学院,扬州 225000
- 2. 扬州大学 广陵学院,扬州 225000
- 折叠
摘要
为了提高短文本匹配精度,本研究提出一种对BERT进行知识增强和结构增强的短文本匹配方法(KS-BERT).该方法首先为输入文本引入外部知识;然后将扩充后的文本同时送往上下文编码器BERT和结构编码器GAT,捕获输入文本的上下文关系特征和结构特征;最后,基于两个特征的融合结果进行匹配判定.基于公开数据集BQ_corpus和LCQMC的实验结果表明,KS-BERT的匹配效果超过了ERNIE2.0等先进模型.本研究工作说明,在面向短文本匹配时,知识增强和结构增强是改进BERT的两种有效途径,在BQ_corpus中分别提升了0.2%和0.3%的准确率,在LCQMC中则分别提升了0.4%和0.9%的准确率.
Abstract
To improve the accuracy of short text matching,a short text matching method with knowledge and structure enhancement for BERT(KS-BERT)was proposed in this study.This method first introduced external knowledge to the input text,and then sent the expanded text to both the context encoder BERT and the structure encoder GAT to capture the contextual relationship features and structural features of the input text.Finally,the match was determined based on the fusion result of the two features.Experiment results based on the public datasets BQ_corpus and LCQMC showed that KS-BERT outperforms advanced models such as ERNIE 2.0.This Study showed that knowledge enhancement and structure enhancement are two effective ways to improve BERT in short text matching.In BQ_corpus,ACC was improved by 0.2%and 0.3%,respectively,while in LCQMC,ACC was improved by 0.4%and 0.9%,respectively.
关键词
深度学习/短文本匹配/图注意力网络/知识增强Key words
Deep learning/Short text matching/Graph attention network/Knowledge enhancement引用本文复制引用
出版年
2024
NETL
NSTL
万方数据