基于多任务深度学习的自适应软参数共享方法
Adaptive Soft Parameter Sharing Method Based on Multi-Task Deep Learning
汪红霞 1金骁 1杜玉坤 1张楠1
作者信息
- 1. 南京审计大学统计与数据科学学院,南京 211815
- 折叠
摘要
在软参数共享模型的基础上,通过任务之间的相似度与参数之间的关系,设置自适应正则项系数λ*,自适应参数衰减比例θ,文章提出了基于多任务深度学习的自适应软参数共享方法.在基于均值约束的L2范数基础上,通过自适应地去除损失函数中正则项中的项数,去除任务间相似度不高的信息.文章的方法将软参数多任务学习动态地转化为软参数多任务与单任务联合学习,相对于软参数多任务学习方法,该方法减少了负迁移现象带来的影响.相对于单任务学习方法,该方法可以极大地降低局部最小解的风险.模拟研究和案例分析都验证了该方法的有效性,该方法的预测精度优于传统的多任务学习和单任务学习.
Abstract
On the basis of the soft parameter sharing model,the authors set the adaptive regular term coefficient λ*and adaptive parameter decay ratio θ by the similarity between tasks and the relationships between parameters.In this paper,the authors propose an adaptive soft parameter sharing method based on multi-task deep learning.On the basis ofL2norm based on the mean constraint,the effect of removing information with low similaritities between tasks can achieve by adaptively removing the number of terms in the regular term of the loss function.The approach in this paper dynamically transforms soft parameter multi-task learning into joint soft parameter multi-task and single-task learning.Compared with soft parameter multi-task learning methods,this method reduces the impact of negative migration phenomena.Compared with single-task learning method,this method can greatly reduce the risk of local minimum solution.Both simulation studies and case analyses have confirmed the effectiveness of this approach,demonstrating that its achieves superior predictive accuracy compared to traditional multi-task learning and single-task learning methods.
关键词
多任务学习/软参数共享/自适应参数衰减比例/自适应正则项系数Key words
Multi-task learning/soft parameter sharing/adaptive parameter decay ratio/adaptive regular term coefficients引用本文复制引用
基金项目
国家社会科学基金(22BTJ021)
国家社会科学基金重大项目(23&ZD036)
国家自然科学基金重点项目(72342019)
国家自然科学基金一般项目(12371267)
江苏省研究生科研创新计划(KYCX22-2154)
出版年
2024
NETL
NSTL
万方数据