BERT sentiment text classification research based on fractional gaussian noise
Due to the large number of parameters in the BERT model and the potential for overfitting during its pre-training phase,this paper proposes a method involving the integration of a plug-and-play module based on Fractional Gaussian Noise(fGn)termed FGnTune.This module utilizes fGn to introduce randomness to improve the effectiveness of the BERT pre-trained model in sentiment text classification tasks.fGn is a form of stochastic signal characterized by long-range dependencies and non-stationarity.The integration of fGn noise into the parameters during the fine-tuning phase of BERT enhances the robustness of the model,thereby mitigating the risk of overfitting.Experimental analyses conducted on various network models and datasets demonstrate that the integration of the FGnTune module leads to a modest improvement in accuracy ranging from 0.3%to 0.9%,without the need for additional model parameters or increasing structural complexity.
Text classificationBERTSentiment textDeep learning
国家科技期刊平台
NSTL
万方数据
