An Adaptive Low-Rank Algorithm for Multi-Task AUC Learning
In recent years,benefiting from the excellent performance and work efficiency of deep neural networks(DNNs),machine learning technology has achieved great success in various fields,such as natural language processing,computer vision,medical named entity recognition,and medical image analysis.In this field,multi-task learning(MTL)is based on the correlation between similar tasks for learning transfer,enabling the model to still exhibit good generalization performance in scenarios with insufficient data.In the past decade,most existing methods are proposed for the balanced category distribution,and use accuracy-based metrics as the benchmark evaluations.However,many practical applications,such as disease detection and spam,suffer from imbalanced sample distributions,which causes the performance degradation of DNNs.Furthermore,multi-task learning has high requirements for task relevance and is apt to the negative transfer phenomenon.Specifically,when models learn to share knowledge among tasks,the irrelevant knowledge may mislead the model training in the wrong direction.This process would result in an unexpected dilemma while most existing methods cannot be effectively applied in such scenarios.Hence,to address this learning problem,designing a multi-task learning algorithm that can learn in imbalanced sample scenarios with low-correlation tasks is of paramount importance to practical applications,as well as represents a critical machine learning challenge.This paper proposes a multi-task AUC optimization method based on an adaptive low-rank Factor Nuclear Norm minus Frobenuis Norm(FNNFN)regularizer to achieve robustness on imbalanced and irrelevant data,doomed MTAUC-FNNFN.Firstly,the area under the ROC curve(AUC),which is usually adopted as the measure for imbalanced distribution,is introduced for directly reflecting the model performance among tasks.Considering the discontinuity and non-differentiability of the loss function for AUC,this work establishes a novel multi-task learning algorithm for AUC optimization,which greatly improves the AUC value in imbalanced scenarios.Meanwhile,in order to effectively optimize,this method reconstructs the original pairwise AUC formulation into an instance-wise minimax optimization problem,reducing the complexity of per-iteration from O(Lni,+ni,-)to O(L(ni,++ni,-)).On top of this well-formed optimization objective,the factor parameters could be easily updated with the gradient descent ascent method.For resisting the negative effect of irrelevant tasks,this paper further introduces an adaptive low-rank regulari-zation term FNNFN to eliminate negative transfer phenomena in multi-task learning and improve the generalization performance of the model.Specifically,penalizing the small singular values empirically equates to dropping the trivial data.In this case,this low-rank structure remains the relevant information within the matrix parameters for sharing knowledge.For the purpose of achieving a comprehensive assessment,we make a comparison between the proposed method and other methods including multi-task learning methods,AUC optimization methods,and low-rank representation methods.For fairness,we uniformly apply them to multi-task datasets and estimate their performance with the AUC metric.A series of experimental results of our method on four simulated datasets and three actual datasets,Landmine,MHC-I,and USPS,consistently demonstrate the effectiveness of our proposed algorithm.
万方数据
维普
