首页|基于帧内-帧间自融合的双流泛化人脸伪造检测方法

基于帧内-帧间自融合的双流泛化人脸伪造检测方法

扫码查看
原文链接
  • 国家科技期刊平台
  • NETL
  • NSTL
  • 万方数据
现有人脸伪造检测方法往往在已知伪造类型上表现良好,但面对未知数据时检测性能有所下降,模型易受到过拟合的影响,检测泛化性不足.针对此问题,提出一种基于帧内-帧间自融合的双流泛化人脸伪造检测方法,从数据增强和检测器改进2个方面提高检测泛化性.设计帧内-帧间自融合模块,分别利用同帧人脸、帧间人脸进行数据增强:帧内自融合子模块利用同帧人脸生成训练数据,从而避免人脸图像身份信息干扰;帧间自融合子模块利用伪造视频的帧间不一致性,进一步构造多样性丰富、逼真的训练数据集,从而有效防止模型的过拟合,确保检测模型的泛化能力.此外,设计基于通道注意力机制的双流特征融合网络,在网络的浅层提取RGB特征、高频特征并进行融合来挖掘伪造信息,在提升模型性能的同时缓解网络的参数增长.将模型在4个数据集上与9种主流检测方法进行对比实验,结果表明:在跨数据集实验中,所提方法较次优方法AUC均值提高1.52个百分点,EER均值降低1.5个百分点;在跨伪造方法实验中,所提方法在4种伪造方法子数据集上均取得最优或次优效果.实验结果验证了该方法优秀的泛化能力.
Dual-Stream Generalized Face Forgery Detection Method Based on Intra-Inter Frame Self-Blending
Although existing methods for detecting face forgery perform well within familiar source domains,they often suffer from overfitting,leading to a lack of generalizability in face forgery detection.As a result,their performance significantly decreases when faced with unfamiliar or unknown scenarios.To address this issue,this study proposes a dual-stream generalized face forgery detection method based on intra-inter frame self-blending.The intra-inter frame self-blending module is designed to prevent detector overfitting from unrelated identity information by leveraging inconsistencies within and between frames to generate a diverse and realistic forgery training set.This method enhances the generalizability of the detection model.Additionally,a detection model is developed as a dual-stream network using an RGB-frequency feature-enhancing module,which extracts and fuses RGB and high-frequency features within the shallow layers of the network to capture forged artifacts.This method not only enhances the model performance but also alleviates the increase in model parameter size.Experiments are conducted against nine mainstream methods across four datasets,with the proposed model improving the AUC by 1.52％and EER by 1.5％on average in cross-dataset experiments.In addition,it ranks first or second in all four sub-datasets of different manipulations in cross-manipulation experiments.These results demonstrate that the proposed method achieves excellent generalizability in face forgery detection.

face forgery detectioninter-intra frame self-blendingfeature fusionattention mechanismdual-stream networkgeneralization ability

董丰恺、邹晓强、王佳慧、马利民、杨文元、刘熙尧

展开 >

中南大学计算机学院,湖南长沙 410083

国家信息中心网络安全部,北京 100045

北京信息科技大学计算机学院,北京 100101

中山大学网络空间安全学院,广东深圳 518107

展开 >

人脸伪造检测 帧内-帧间自融合 特征融合 注意力机制 双流网络 泛化能力

国家自然科学基金青年基金湖南省自然科学基金面上项目湖南省创新生态建设计划-政策性项目长沙市杰出创新青年培养计划

616025272020JJ47462022GK5002kq2209003

2024

10.19678/j.issn.1000-3428.0068356

计算机工程
华东计算技术研究所　上海市计算机学会

计算机工程

CSTPCD北大核心
影响因子：0.581
ISSN：1000-3428
年,卷(期)：2024.50(10)
  • 4