PromptWE:A fact-checking method based on prompt learning with explanations
[Objective]In the contemporary"We Media"era,the simplification of news production and dissemination has elevated every individual to the status of news producer and disseminator,and a large amount of false information also follows.Despite the increasing and abundant information on the Internet,the regulation of false information is relatively weak.Consequently,fact-checking is becoming more and more important work,while traditional related work tends to simply label predictions without explaining the reason for the label.The generated explanation in a few studies is also relatively primitive which is hard to comprehend.Because Fact-checking demands a substantial amount of common sense,reasoning,and background knowledge about claims.Prompt learning may further utilize common sense and reasoning ability in pre-trained language models.It may also incorporate the relevant information or additional details within the explanation for claims.In all,it is essential to generate high-quality smooth explanations and further leverage generated explanations for improving classification performance through prompt learning.[Methods]To address this multifaceted challenge,we propose the PromptWE model(Prompt With Evidence)that uses the prompt learning paradigm to integrate auto-generated explanations with claims.We not only provide natural language explanations that enhance the explainability of the classification result but also further improve the model performance by combining explanation into prompt learning.The model performs hierarchical evidence distillation on many related new reports for every claim to obtain relevant evidence,then uses the BART-CNN model to summarize these incoherent pieces of evidence into one smooth explanation.Consequently,it integrates the claim and explanation into six self-designed templates for prompt learning.Finally,we ensemble the result from different templates to predict the authenticity of the news.Moreover,we replace the generated explanation with the professional explanation from the dataset to investigate the impact of expert evidence on the prompt learning models.[Results]Our method achieves good results on two fact-checking datasets:Liar-RAW and RAWFC.Its F1 score is 5％higher than the state-of-the-art model on both datasets at least.We also find that ensemble learning with multiple templates can effectively improve the F1 score of the model.For explanation generation,the model has a higher ROUGE-2 score than the former model.After integrating professional evidence into the prompt templates,the model achieves significant improvement in the classification results on the two datasets,with a maximum improvement of 15％when compared to the results of the PromptWE model.Also,we find that for multi-class classification task,the model with integrated professional evidence exhibited exhibits significant performance improvement on more challenging categories,such as half-true and mostly true.[Conclusions]Related experiments indicate that incorporating extracted explanations as supplementary background knowledge about claims,along with the common sense and reasoning abilities learned from pre-trained models,into prompt learning templates can further enhance classification performance for claim veracity.Moreover,sequentially employing the methods of hierarchical evidence extraction and text summarization makes explanations more concise,coherent,and comprehensible.Also,the explanation extracted from unrelated evidence is better suited for integration into prompt learning methods.The further improvement in classification performance after incorporating professional evidence underscores that this approach could swiftly identify accurate and informative prompt templates,facilitating subsequent more efficient utilization of general large models like ChatGPT.
NETL
NSTL
万方数据
