基于异质图神经网络预训练的多标签文档分类研究
Pre-training of Heterogeneous Graph Neural Networks for Multi-label Document Classification
吴家伟 1方全 2胡骏 2钱胜胜2
作者信息
- 1. 郑州大学河南先进技术研究院 郑州 450002
- 2. 中科院自动化所模式识别国家重点实验室 北京 100190
- 折叠
摘要
多标签文档分类是一种将文档实例与相关标签相关联的技术,近年来受到越来越多研究者的关注.现有的多标签文档分类方法尝试探索文本之外的信息的融合,如文档元数据或标签结构.然而,这些方法要么简单地利用元数据的语义信息,要么没有考虑标签的长尾分布,因此忽略了文档及其元数据之间的高阶关系和标签的分布规律等信息,从而影响到多标签文档分类的准确性.因此,文中提出一种新的基于异质图神经网络预训练的多标签文档分类方法.该方法通过构造文档与其元数据的异质图,采用两种对比学习预训练方法捕获文档与其元数据之间的关系,并通过平衡标签长尾分布的损失函数来提高多标签文档分类的准确性.在基准数据集上的实验结果表明,所提方法的准确率比Transformer提高了 8％,比BertXML提高了4.75％,比 MATCH 提高了 1.3％.
Abstract
Multi-label document classification aims to associate document instances with relevant labels,which has received in-creasing research attention in recent years.Existing multi-label document classification methods attempt to explore the fusion of information beyond the text,such as document metadata or label structure.However,these methods either simply use the seman-tic information of metadata or do not consider the long-tail distribution of labels,thereby ignoring higher-order relationships be-tween documents and their metadata and the distribution pattern of labels,which affects the accuracy of multi-label document classification.Therefore,this paper proposes a new multi-label document classification method based on the pre-training of hete-rogeneous graph neural networks.The method constructs a heterogeneous graph based on documents and their metadata,adopts two contrastive pre-training methods to capture the relationship between documents and their metadata,and improves the accura-cy of multi-label document classification by balancing the problem of long-tail distribution of labels through a loss function.Ex-perimental results on the benchmark dataset show that the proposed method outperforms Transformer BertXML and MATCH by 8％,4.75％,1.3％,respectively.
关键词
多标签文档分类/元数据/异质图神经网络/预训练/长尾分布Key words
Multi-label document classification/Metadata/Heterogeneous graph neural network/Pre-training/Long-tail distribu-tion引用本文复制引用
基金项目
国家自然科学基金(62072456)
国家自然科学基金(62036012)
国家自然科学基金(62106262)
出版年
2024
NETL
NSTL
万方数据