基于分数阶高斯噪声的BERT情感文本分类研究
BERT sentiment text classification research based on fractional gaussian noise
龙雨欣 1蒲亦非 1张卫华1
作者信息
- 1. 四川大学计算机学院,成都 610065
- 折叠
摘要
由于BERT模型庞大的参数量和在预训练阶段的过拟合问题,本文针对性地提出了基于分数阶高斯噪声(fGn)的即插即用模块FGnTune.该模块利用fGn引入随机性,用于提高BERT预训练模型在情感文本分类任务中的性能.fGn是具有长程依赖和非平稳性的随机信号,通过在BERT微调阶段为参数融入fGn噪声,进一步增强模型的鲁棒性,降低过拟合的可能性.通过对不同网络模型及多种数据集进行实验分析,在不需增加模型的额外参数或增加其结构复杂度的前提下,引入FGnTune模块可以使模型的准确率在原有基础上提升约0.3%～0.9%.
Abstract
Due to the large number of parameters in the BERT model and the potential for overfitting during its pre-training phase,this paper proposes a method involving the integration of a plug-and-play module based on Fractional Gaussian Noise(fGn)termed FGnTune.This module utilizes fGn to introduce randomness to improve the effectiveness of the BERT pre-trained model in sentiment text classification tasks.fGn is a form of stochastic signal characterized by long-range dependencies and non-stationarity.The integration of fGn noise into the parameters during the fine-tuning phase of BERT enhances the robustness of the model,thereby mitigating the risk of overfitting.Experimental analyses conducted on various network models and datasets demonstrate that the integration of the FGnTune module leads to a modest improvement in accuracy ranging from 0.3%to 0.9%,without the need for additional model parameters or increasing structural complexity.
关键词
文本分类/BERT/情感文本/深度学习Key words
Text classification/BERT/Sentiment text/Deep learning引用本文复制引用
基金项目
国家自然科学基金面上项目2022―2025(62171303)
出版年
2024
国家科技期刊平台
NSTL
万方数据