首页|Improving Transformer with Sequential Context Representations for Abstractive Text Summarization
Improving Transformer with Sequential Context Representations for Abstractive Text Summarization
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NETL
Springer Nature
Recent dominant approaches for abstractive text summarization are mainly RNN-based encoder-decoder framework， these methods usually suffer from the poor semantic representations for long sequences。 In this paper， we propose a new abstractive summarization model， called RC-Transformer (RCT)。 The model is not only capable of learning long-term dependencies， but also addresses the inherent shortcoming of Transformer on insensitivity to word order information。 We extend the Transformer with an additional RNN-based encoder to capture the sequential context representations。 In order to extract salient information effectively， we further construct a convolution module to filter the sequential context with local importance。 The experimental results on Gigaword and DUC-2004 datasets show that our proposed model achieves the state-of-the-art performance， even without introducing external information。 In addition， our model also owns an advantage in speed over the RNN-based models。
TransformerAbstractive summarization Introduction
Tian Cai、Mengjun Shen、Huailiang Peng、Lei Jiang、Qiong Dai
展开 >
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China,School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
CCF international conference on natural language processing and Chinese computing
NETL
Springer Nature
