面向自然语言处理的词向量模型研究综述
Survey of Word Vector Model for Natural Language Processing
安俊秀 1蒋思畅1
作者信息
- 1. 成都信息工程大学 软件工程学院,四川 成都 610225
- 折叠
摘要
从20 世纪50 年代至今,自然语言处理(Natural Language Processing,NLP)取得了长足的发展.早期的词向量模型证明了对该领域的研究需要使用数学方法,而不是人类的语言规则.进入21 世纪后,静态模型以深度学习技术为基础,在很多任务中取得了不错的表现;动态模型再将预训练技术融入进来,实现了根据语境对词向量进行调整的功能,为NLP领域带来了里程碑式的突破,后续研究在此基础上向各领域延伸扩展,并且在现实生活中得到了大规模的应用.文章首先对词向量模型及其发展历史做了介绍,然后分析了现代的词向量模型(NNLM,Word2Vec,FastText,Glove,ELMo,GPT,BERT),其次说明了多种基于预训练技术的扩展模型和当前自然语言处理技术的应用现状,最后总结了目前存在的主要问题,并提出对未来研究的展望.
Abstract
Since the1950s,Natural Language Processing(NLP)has made great progress.The early word vector model demonstrates that the study of NLP requires mathematical methods rather than human language rules.After entering the 21st century,the static model which based on deep learning techniques achieves good performance in many tasks.The dynamic model makes use of pre-training techniques and realizes the function of adjusting word vectors according to the context,which brings a milestone breakthrough in the field of NLP.On this basis,the follow-up research extends to various fields,and has been applied on a large scale in real life.We firstly introduce the word vector model and its development history,then analyze the modern models based on deep learning(NNLM,Word2Vec,FastText,Glove,ELMo,GPT,BERT).Secondly,we explain a variety of extended models based on pre-training technology,and describe the current application status of natural language processing technology.Finally,we summarize the main problems at present,and put forward the prospect of future research.
关键词
自然语言处理/词向量/深度学习/预训练技术/静态模型/动态模型Key words
natural language processing/word vector/deep learning/pre-training technique/static model/dynamic model引用本文复制引用
出版年
2023
国家科技期刊平台
NSTL
维普
万方数据