基于异质图神经网络的知识增强文本理解研究
Research on Knowledge Enhanced Text Understanding Based on Heterogeneous Graph Neural Network
凌骏 1赵前 2张帆 1李倩倩1
作者信息
- 1. 上海电气电站集团 上海 201199
- 2. 华东师范大学 上海 200062
- 折叠
摘要
针对当下流行的语言模型如BERT,即使其在大量文本下预训练而具有了较好的文本理解能力,但由于文本中知识共现的稀疏性导致模型难以理解复杂语义的场景的问题,提出一种引入知识图谱结合异质图网络增强文本理解的方法.该方法根据文本中的实体构造异质图,并基于边采样机制生成子视图.引入R-GCN聚合异质子视图下的邻居信息,并约束同节点在不同视图下基于相似性的度量损失.最终得到知识增强的文本表示.在公开数据集WinoGrande上的结果表明,该方法在准确率指标下有着明显的提升,相比于未增强时提升了1.3%.
Abstract
Nowadays,popular language models such as BERT have good reading comprehension ability because of the pre-training of large-scale corpus.However,due to the sparseness of knowledge co-occurrence in the sentences,it is difficult for the model to understand scenes with complex semantics.To solve this problem,a method of introducing knowledge graph combined with heterogeneous graphs to enhance reading comprehension is proposed.This method constructs heterogeneous graphs based on the entities in the sentence,and generates sub-views based on the edge-dropout.The R-GCN aggregates the neighbor information in the heterogeneous sub-views,and constrains the similarity-based metric loss of the same node in different views.Finally,the knowledge-enhanced textual representation is obtained.The results on the WinoGrande show that this method has a significant im-provement in accuracy,which is 1.3%higher than without knowledge enhancement.
关键词
知识图谱/异质图神经网络/预训练语言模型/文本理解/自然语言处理Key words
knowledge graph/heterogeneous graph neural network/pre-trained language model/reading comprehension/natural language processing引用本文复制引用
基金项目
国家自然科学基金项目(61773167)
上海市科委项目(19511120200)
出版年
2024
国家科技期刊平台
NSTL
万方数据