基于数据表示不变性的域泛化研究
Domain generalization based on data representation invariance
倪云昊 1黄雷1
作者信息
- 1. 北京航空航天大学人工智能研究院,北京 100191;北京航空航天大学复杂关键软件环境全国重点实验室,北京 100191
- 折叠
摘要
域泛化是人工智能近几年非常热门的一个研究方向,希望在不同的数据分布中学习到与任务相关的不变表征,即移除不同域在学习任务中的影响,从而提升模型的域泛化能力.为提升模型域泛化能力,利用基于不变性风险最小化的思想,具体将神经网络分为特征提取器和不变性分类器进行分别探究.在特征提取器上,采用了基于牛顿迭代的组白化方法来控制激活值的分布,从而使得不同的图像经过神经网络后能够去除部分域信息,以求达到域泛化的目的;在不变性分类器上,探究了特征和权重的规范化方法对模型域泛化效果的影响,并提出了基于余弦相似度损失函数的雪花算法,该算法提升了模型域泛化的准确率.此外,提供了关于雪花算法的理论推导并做了深入分析,为实验提供了理论支撑.
Abstract
Domain generalization has become a prominent research direction in artificial intelligence,aiming to learn task-related invariant representations from different data distributions.It seeks to remove the impact of varying domains on learning tasks,thereby enhancing the model's domain generalization capabilities.Based on the idea of minimizing the risk of invariance,this paper divided neural networks into feature extractors and invariance classifiers for exploration.For the feature extractor,a group whitening method based on Newtonian iteration was utilized to control the distribution of activation values.This allowed different images to remove part of the domain information after passing through the neural network,thus achieving the purpose of domain generalization.For the invariance classifier,the effects of the normalization method of features and weights on the generalization effect of the model domain were explored,and a snowflake algorithm based on the cosine similarity loss function was proposed.This algorithm improved the accuracy of model domain generalization.In addition,extensive theoretical derivations about the snowflake algorithm and in-depth analyses were provided,offering sufficient theoretical support for the experiment.
关键词
域泛化/不变风险最小化/组白化/迭代白化/雪花算法Key words
domain generalization/invariant risk minimization/group whitening/iterative whitening/snowflake algorithm引用本文复制引用
基金项目
科技创新2030新一代人工智能重大项目(2021ZD0112901)
国家自然科学基金项目(62106012)
中央高校基本科研业务费专项资金()
出版年
2024
NETL
NSTL
万方数据