Abstractive Text Summarization Method Incorporating Convolutional Shrinkage Gating
Driven by deep learning techniques,Sequence to Sequence(Seq2Seq)model,based on an encoder-decoder architecture combined with an attention mechanism,is widely utilized in text summarization research,particularly for abstractive text summarization tasks.Remarkable results are achieved by this model.However,limitations are faced by existing models using Recurrent Neural Network(RNN),such as insufficient parallelism,low time efficiency,and a tendency to produce summaries that are either redundant,repetitive,or semantically irrelevant.Additionally,these models often fail to fully summarize useful information and ignore the connection between words and sentences.In response to these challenges,a text summarization method based on Transformer and convolutional shrinkage gating is proposed.Different levels of text representations are extracted using BERT as an encoder,which then obtains contextual encoding.The convolutional shrinkage gating unit is adopted to adjust encoding weights,strengthen global relevance,remove interference from useless information,and obtain the final encoding output after filtering.Three different decoders are designed:the basic Transformer decoding module,the decoding module with a shared encoder,and the decoding module using GPT.These are aimed at strengthening the association between encoder and decoder and exploring model structures capable of generating high-quality abstracts.Evaluation scores of the TCSG,ES-TCSG,and GPT-TCSG models in this method are shown to increment by no less than 1.0 on both LCSTS and CNNDM datasets,verifying the validity and feasibility of the method relative to mainstream benchmark models.
abstractive text summarizationSequence to Sequence(Seq2Seq)modelTransformer modelBERT encoderconvolutional shrinkage gating unitdecoder
国家科技期刊平台
NSTL
万方数据
维普
