基于多通道残差混合空洞卷积的注意力词义消歧
An Attention Word Sense Disambiguation Model Based on Multi-Channel Residual Hybrid Dilated Convolution
张春祥 1张育隆 1高雪瑶1
作者信息
- 1. 哈尔滨理工大学计算机科学与技术学院,哈尔滨 150080
- 折叠
摘要
为了提高词义消歧模型的泛化能力和准确率,增强其在自然语言处理任务中的应用效果,提出了基于多通道残差混合空洞卷积注意力机制的词义消歧模型.利用语言学知识构建消歧特征,通过3种向量化方式生成三通道词嵌入矩阵,将位置编码与词嵌入矩阵深度融合,并设计了复杂卷积编码器,以增强模型的表达能力.在词义消歧数据集SemEval-2007:Task#5和SemEval-2021:Task#2上的实验结果表明,用所提方法在置信区间的平均偏差上较基于聚类语义标签和多头注意力机制的模型分别降低了 1.345％和2.157％,有效提升了词义消歧性能.
Abstract
Aiming at the insufficient generalization ability of the current word sense disambiguation model,multi-channel residual hybrid dilated convolution with attention word sense disambiguation model is proposed to improve disambiguation accuracy and provide help for more tasks of natural language processing.Linguistic knowledge is used to construct disambiguation features,three vectorization methods are employed to vectorize the disambiguation features to form a three-channel word embedding matrix,and positional coding is deeply fused with the three-channel word embedding matrix.A complex convolutional encoder is designed to increase the expressive power of the model.Experiments are conducted on word sense disambiguation datasets SemEval-2007:Task#5 and SemEval-2021:Task#2 in tasks of semantic evaluation,and the results show that compared to the latest word sense disambiguation using clustered sense labels and multi-head attention mechanism,the average bias of the proposed method is reduced to 1.345％ and 2.157％ respectively,which effectively improves the performance of word sense disambiguation.
关键词
词义消歧/语言学知识/混合空洞卷积/卷积编码器Key words
word sense disambiguation/linguistic knowledge/hybrid dilated convolution/convolutional encoder引用本文复制引用
出版年
2024
NETL
NSTL
万方数据