[Purpose/Significance]To enhance the semantic accuracy and diversity in data augmentation methods for multimodal rumor detection,exploring models and methods that have the potential to enhance the detection performance can contribute to the identification of online rumors,as well as to the reinforcement of network information governance capabilities.[Method/Process]A multimodal rumor detection model named TARD-GPT-4 was proposed,which leveraged GPT-4 for data augmentation.The model employed BERT and ViT models to extract textual and visual features,respectively.A supervised contrastive learning strategy was used to further explore the label attribute features.Finally,a full connected layer was used for rumor detection discrimi-nation.[Result/Conclusion]Incorporating supervised contrastive learning and prompting large language models using rephrasing method to augment data have a positive effect on improving the accuracy of multimodal rumor detection.Compared to the optimal baseline model,TARD-GPT-4 achieves a 1.62％higher accuracy in multi-modal rumor detection.The experimental part also investigates the impact of various data augmentation methods and finds that prompting LLMs for paraphrasing yields the most favorable results.
关键词
数据增强/对比学习/多模态/谣言检测
Key words
data augmentation/contrastive learning/multimodal/rumor detection
维普
万方数据