融合义原相似度矩阵与字词向量双通道的短文本语义匹配策略
Short Text Semantic Matching Strategy Fusing Sememe Similarity Matrix and Dual-channel of Char-Word Vectors
刘东旭 1段利国 2崔娟娟 1常轩伟1
作者信息
- 1. 太原理工大学计算机科学与技术学院 山西晋中 030600
- 2. 太原理工大学计算机科学与技术学院 山西晋中 030600;山西电子科技学院 山西临汾 041000
- 折叠
摘要
短文本语义匹配任务的目的是判断两个短文本句子的语义是否一致.然而,现有的许多方法往往存在短文本语义信息不足、无法有效识别同义词等问题.针对这些不足,提出一种融合义原相似度矩阵与字词向量双通道的短文本语义匹配策略.首先,利用预训练模型Bert对输入的句子对进行编码;然后,对于句子中词级别的语义信息,利用FastText模型训练并获取文本的词向量,并加入BiLSTM模型进一步提取上下文语义信息.为了有效利用义原信息,在上述的双通道中分别加入多头注意力和用于对分离向量进行交互计算的协同注意力,并在注意力中分别融入对应的义原相似度矩阵,最后综合上述两部分向量推断出语义的一致性.在金融领域数据集BQ和开放域数据集LCQMC上的实验证明了所提算法的有效性.
Abstract
The purpose of the short text semantic matching task is to judge whether the semantics of two short text sentences are consistent.However,many existing methods often have shortcomings such as insufficient semantic information of short text and inability to effectively identify synonyms.In response to these shortcomings,this paper proposes a short text semantic matching strategy that fuses sememe similarity matrix and dual-channel of char-word vectors.Firstly,the pre-trained model Bert is used to encode the input sentence pairs;for the word-level semantic information in the sentence,the FastText model is used to train and obtain the word vector of the text,and the BiLSTM model is added to further extract the contextual semantic information.Se-condly,making effective use of the semantic information,multi-head attention and co-attention for interactive calculation of sepa-ration vectors are added to the above-mentioned dual-channel.And the semantic similarity matrix is integrated into the attentions respectively.Finally,infer the semantic consistency according to the above vectors.The effectiveness of the above algorithm is proved by experiments on the financial dataset BQ and the open domain dataset LCQMC.
关键词
自然语言处理/短文本/义原/协同注意力/字词向量Key words
Natural language processing/Short text/Sememe/Co-attention/Char-Word vector引用本文复制引用
出版年
2024
NETL
NSTL
万方数据