By providing a comprehensive review of Transformer-based pre-training models,this paper explored their practical applications and potential advantages.With the increasing attention in fields such as natural language processing,computer vision,speech processing,and interdisciplinary application,this technology has gained significant importance.The aim is to summarize the latest progress and existing issues of this technology through a comprehensive analysis of existing literature.It adopted a sys-tematic literature review methodology to conduct an extensive survey and analysis of relevant literature.It provided a review of Transformer-based pre-training models,and pointed out future research directions and priorities.
关键词
Transformer/预训练模型/自然语言处理/语音处理/学科交叉应用
Key words
Transformer/pre-training model/natural language processing/speech processing/interdisciplinary application
万方数据