首页|基于注意力掩码的领域泛化研究

基于注意力掩码的领域泛化研究

扫码查看
深度学习在区分特征方面表现较好,但将其应用于未知领域时,已训练好的模型往往会因领域移位而导致模型性能下降.针对该情况,领域泛化(DG)从多个源领域学习可迁移特征,将其泛化到未知的目标领域.由于不同领域训练的模型更偏向其中最显著的特征,往往会忽略与任务相关的一般性特征,而可迁移特征通常并不是该领域最显著的特征.因此,从这个角度提出一种基于注意力掩码来屏蔽特征的正则化方法,通过注意力掩码模块生成注意力掩码,对权重高的特征进行屏蔽,以提升模型泛化性能.实验表明,在3个基准数据集上测试的精度相较于基线模型分别提升2.6%、2.0%、4.2%,证明该方法既能提升模型在未知领域上的性能,也体现了其在领域泛化数据集上的普适性.
Domain Generalization Research Based on Attention Mask
Deep learning performs well in distinguishing features,but when applied to unknown domains,trained models often experience performance degradation due to domain shift.In response to this situation,Domain Generalization(DG)learns transferable features from mul-tiple source domains and generalizes them to unknown target domains.Due to the bias of models trained in different fields towards the most prominent features,they often overlook general features related to the task,and transferable features are usually not the most prominent fea-tures in that field.Therefore,from this perspective,a regularization method based on attention masks is proposed to mask features,which gen-erates attention masks through the attention mask module to mask high weight features and improve the model's generalization performance.The experiment showed that the accuracy tested on three benchmark datasets increased by 2.6%,2.0%,and 4.2%compared to the baseline model,respectively,proving that this method can not only improve the performance of the model in unknown domains,but also reflect its uni-versality on domain generalization datasets.

domain generalizationtransfer learningattention mechanismdeep learningregularization

路京、沈洋、许浩、包艳霞、应震

展开 >

浙江理工大学 计算机科学与技术学院(人工智能学院),浙江 杭州 310020

丽水学院 工学院,浙江 丽水 323000

浙江掌信传媒科技有限公司,浙江 丽水 323020

浙江聚新自动化设备有限公司,浙江 丽水 323010

展开 >

领域泛化 迁移学习 注意力机制 深度学习 正则化

浙江省自然科学基金项目丽水市公益性技术应用研究计划项目

LY21F020042022GYX12

2024

软件导刊
湖北省信息学会

软件导刊

影响因子:0.524
ISSN:1672-7800
年,卷(期):2024.23(4)
  • 26