一种特征融合的双流深度检测伪造人脸方法
A Feature Fusion Dual-Stream Deepfake Detection Method for Forged Faces
孟媛 1汪西原2
作者信息
- 1. 宁夏大学 电子与电气工程学院,宁夏 银川 750021
- 2. 宁夏大学 电子与电气工程学院,宁夏 银川 750021;宁夏沙漠信息智能感知重点实验室,宁夏 银川 750021
- 折叠
摘要
Deepfake技术的迅速发展,使得深度伪造视频和音频内容日益逼真,这种技术被广泛应用于政治伪造、金融欺诈和虚假新闻传播等领域.因此,研究和开发高效的Deepfake检测方法变得尤为关键.本研究探索了一种结合ViT与CNN的策略,充分利用CNN在局部特征提取方面的优势,以及ViT在建模全局关系方面的潜力,以提升Deepfake检测算法在实际应用中的效能.此外,为增强模型对图像或视频压缩引起的影响的抵御能力,引入频域特征,使用双流网络提取特征,以提高模型在跨压缩场景下的检测性能和稳定性.实验结果表明,基于多域特征融合的双流网络模型在FaceForensics++数据集上有较好的检测性能,其ACC值达96.98%、AUC值达98.82%.在跨数据集检测方面也取得了令人满意的结果,在Celeb-DF数据集上的AUC值达75.41%.
Abstract
The rapid advancement of Deepfake technology has rendered deepfake video and audio content increasingly realistic,with widespread applications in political forgery,financial fraud,and the dissemination of fake news.Therefore,the research and development of efficient Deepfake detection methods have become cru-cial.This study explores a strategy that combines Vision Transformers(ViT)with Convolutional Neural Net-works(CNN),leveraging the advantages of CNN in local feature extraction and the potential of ViT in model-ing global relationships to enhance the performance of Deepfake detection algorithms in practical applications.Moreover,to strengthen the model's resilience against the impacts of image or video compression,frequency domain features are introduced,and a dual-stream network is employed to extract features,thereby improving detection performance and stability across compressed scenarios.Experimental results indicate that the dual-stream network model based on multi-domain feature fusion demonstrates commendable detection performance on the FaceForensics++dataset,achieving an ACC value of 96.98% and an AUC value of 98.82% .Satis-factory results are also obtained in cross-dataset detection,with an AUC value of 75.41% on the Celeb-DF dataset.
关键词
Deepfake检测/CNN结合ViT/RGB频域特征融合/跨压缩场景Key words
Deepfake detection/CNN combined with ViT/RGB frequency domain feature fusion/cross-compression scenarios引用本文复制引用
基金项目
国家自然科学基金资助项目(42361056)
出版年
2024
国家科技期刊平台
NSTL
万方数据