Short Text Classification Combining Attention Mechanism and Mengzi Model
How to use short text classification technology to mine useful text information is one of the current hot research direc-tions.To solve the problem of sparse feature information and difficult extraction of short text,a short text classification model named Mengzi-ADCBU is proposed.This model uses Mengzi pre-training model to convert input text information into correspond-ing text representation.Then,the obtained text vectors are input to the improved deep pyramid convolutional neural network and the bidirectional gated unit integrated with multi-head attention mechanism to extract text feature information,and the extracted feature information is fused and sent to the full connection layer and Softmax function to complete short text classification.Multiple models comparison experiments are carried out on the publicly available THUCNews short text data set and SougouCS short text data set respectively.The experimental results show that the proposed Mengzi-ADCBU model is better than the current mainstream models in the accuracy,precision,recall rate and F1 value of short text classification and has better short text classification ability.
short textmulti-head attentiondeep pyramid convolutional neural netwrksbidirectional gated unit
国家科技期刊平台
NSTL
万方数据
