基于Transformer模型的文本自动摘要生成
Automatic Text Summary Generation Based on Transformer Model
刘志敏 1张琨 1朱浩华1
作者信息
摘要
论文探讨文本摘要的自动生成技术,其任务是产生能够表达文本主要含义的简明摘要.传统的Seq2Seq结构模型对长期特征和全局特征的捕获和存储能力有限,导致所生成的摘要中缺乏重要信息.因此,论文基于Transformer模型提出了一种新的生成式文本摘要模型RC-Transformer-PGN(RCTP).该模型首先使用了一个附加的基于双向GRU的编码器来扩展Transformer模型,以捕获顺序上下文表示并提高局部信息的捕捉能力,其次引入指针生成网络以及覆盖机制缓解未登录词和重复词问题.在CNN/Daily Mail数据集上的实验结果表明论文模型与基线模型相比更具竞争力.
Abstract
This paper discusses the automatic generation technology of text summarization,whose task is to generate a concise summary which can express the main meaning of text.The traditional Seq2Seq structural model has limited ability to capture and store long-term features and global features,resulting in a lack of important information in the generated abstract.Therefore,this paper proposes a new abstractive summarization model called RC-Transformer-PGN(RCTP)based on the Transformer model.The model first uses an additional encoder based on bidirectional GRU to improve the Transformer model to capture sequential context representation and improve the ability to capture local information.Secondly,it introduces Pointer Generation Network and Cover-age mechanism to alleviate the problem of Out-Of-Vocabulary words and repeated words.The experimental results on CNN/Daily Mail dataset show that our proposed model is more effective than the baseline model.
关键词
生成式文本摘要/Transformer模型/指针生成网络/覆盖机制Key words
abstractive summarization/Transformer model/pointer generator network/coverage mechanism引用本文复制引用
出版年
2024
国家科技期刊平台
NSTL
万方数据