Aiming at the problem that current short text classification only uses classification labels as the basis for judging classification re-sults,and ignores the semantic information contained in the classified label text,a short text classification method based on label aware atten-tion is proposed,which is based on a large-scale pre trained language model.This method represents text data in distributed vector form through large-scale pre trained language models to obtain richer semantic information;At the same time,incorporating classification label in-formation into the text data training process,using attention mechanisms to make the text data perceive and classify the most relevant informa-tion;Using CNN networks and max pooling layers to extract local word level vector features,in order to better address semantic issues such as double negation and comparative negation in English texts;Using residual connections to fuse sentence level vectors with word level vectors ef-fectively alleviates the problem of text information decay.Tests were conducted on three common English datasets,R8,R52,and MR,and the experimental results showed that the proposed method achieved accuracies of 98.51%and 97.10%on the R8 and R52 datasets,respective-ly,which are better than DeBERTa and BertGCN.
关键词
短文本分类/CNN/标签感知/注意力/预训练
Key words
short text classification/CNN/label awareness/attention/pre-trained
维普
万方数据