Prophecy Distillation for Boosting Abstractive Summarization

扫码查看

原文链接

NETL

外文摘要：Abstractive summarization models learned with maximum likelihood estimation (MLE) have long been guilty of generating unfaithful facts alongside ambiguous focus。 Improved paradigm under the guidance of reference-identified words， i。e。， guided summarization， has exhibited remarkable advantages in overcoming this problem。 However， it suffers limited real applications since the prophetic guidance is practically agnostic at inference。 In this paper， we introduce a novel teacher-student framework， which learns a regular summarization model to mimic the behavior of being guided by prophecy for boosting abstractive summaries。 Specifically， by training in probability spaces to follow and distinguish a guided teacher model， a student model learns the key to generating teacher-like quality summaries without any guidance。 We refer to this process as prophecy distillation， and it breaks the limitations of both standard and guided summarization。 Through extensive experiments， we show that our method achieves new or matched state-of-the-art on four well-known datasets， including ROUGE scores， faithfulness， and saliency awareness。 Human evaluations are also carried out to evidence these merits。 Furthermore， we conduct empirical studies to analyze how the hyperparameters setting and the guidance choice affect TPG performance。

外文关键词：

Abstractive summarizationGuided summarizationKnowledge distillation

作者：

Jiaxin Duan、Fengyu Lu、Junfei Liu

展开 >

作者单位：

School of Software and Microelectronics, Peking University Beijing, China

会议名称：

Joint International Conference on Computational Linguistics, Language Resources and Evaluation

会议地点：

Torino(IT)

会议母体文献：

2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation

页码：

13257-13268

出版时间：

2024