Research on Multimodal Rumor Detection Based on GPT-4 Text Augmentation and Contrastive Learning
[Purpose/Significance]To enhance the semantic accuracy and diversity in data augmentation methods for multimodal rumor detection,exploring models and methods that have the potential to enhance the detection performance can contribute to the identification of online rumors,as well as to the reinforcement of network information governance capabilities.[Method/Process]A multimodal rumor detection model named TARD-GPT-4 was proposed,which leveraged GPT-4 for data augmentation.The model employed BERT and ViT models to extract textual and visual features,respectively.A supervised contrastive learning strategy was used to further explore the label attribute features.Finally,a full connected layer was used for rumor detection discrimi-nation.[Result/Conclusion]Incorporating supervised contrastive learning and prompting large language models using rephrasing method to augment data have a positive effect on improving the accuracy of multimodal rumor detection.Compared to the optimal baseline model,TARD-GPT-4 achieves a 1.62％higher accuracy in multi-modal rumor detection.The experimental part also investigates the impact of various data augmentation methods and finds that prompting LLMs for paraphrasing yields the most favorable results.
data augmentationcontrastive learningmultimodalrumor detection
万方数据
维普
