首页|一种面向中文自动问答的注意力交互深度学习模型

一种面向中文自动问答的注意力交互深度学习模型

扫码查看
随着互联网、大数据的飞速发展,以深度神经网络(DNN)为代表的人工智能技术迎来了黄金发展时期,自动问答作为人工智能领域的一个重要分支,也得到越来越多学者的关注.现有网络模型可以提取问题或答案的语义特征,但其一方面忽略了问题与答案之间的语义联系,另一方面也不能从整体上把握问题或答案内部所有字符之间的潜在联系.基于此,提出了两种不同形式的注意力交互模块,即互注意力交互模块和自注意力交互模块,并设计出一套基于所提注意力交互模块的深度学习模型,用于证明该注意力交互模块的有效性.首先将问题和答案中的每个字符映射成固定长度的向量,分别得到问题和答案对应的字嵌入矩阵;然后将字嵌入矩阵送入注意力交互模块,得到综合考虑问题与答案所有字符之后的字嵌入矩阵,并与之前的字嵌入矩阵相加,送入深度神经网络模块,用于提取问题与答案的语义特征;最后得到问题与答案的向量表示并计算两者之间的相似度.实验结果表明,所提模型的Top-1准确度较主流深度学习模型最高提升了 3.55%,证明了所提注意力交互模块对于改善上述问题的有效性.
Attentional Interaction-based Deep Learning Model for Chinese Question Answering
With the rapid development of the Internet and big data,artificial intelligence,represented by deep neural network(DNN),has ushered in a golden period of development.As an important branch in the field of artificial intelligence,question an-swering has attracted more and more scholars'attention.The existing deep neural network module can extract the semantic fea-tures of the question or answer,however,on the one hand,it ignores the semantic relation between the question and answer,on the other hand,it cannot grasp the potential relation among all the characters in the question or answer as a whole.Therefore,two different forms of attention interaction module,namely cross-embedding and self-embedding,are used to solve the above pro-blems,and a set of deep learning model based on the proposed attention interaction module is designed to prove the effectiveness of this attention interaction module.Firstly,each character in the question and answer is mapped into a fixed length vector,and the corresponding character embedding matrix is obtained respectively.After that,the character embedding matrix is sent into the attentional interaction module to obtain the character embedding matrix that takes all characters of the question and answer into account.After adding the previous character embedding matrix,it is sent into the deep neural network module to extract the se-mantic features of the question and answer.Finally,the vector representations of the question and the answer are obtained,and the similarity between them is calculated.Experiments show that the accuracy of Top-1 of the proposed model is 3.55%higher than that of the mainstream deep learning model at most,which proves the effectiveness of the proposed attention interaction module in resolving the above problems.

Artificial intelligenceQuestion answeringDeep learningAttentionCharacter embedding

蒋锐、杨凯辉、王小明、李大鹏、徐友云

展开 >

南京邮电大学通信与信息工程学院 南京 210003

人工智能 自动问答 深度学习 注意力 字嵌入

国家自然科学基金

62271266

2024

计算机科学
重庆西南信息有限公司(原科技部西南信息中心)

计算机科学

CSTPCD北大核心
影响因子:0.944
ISSN:1002-137X
年,卷(期):2024.51(6)
  • 33