Improvement of Text Similarity Algorithm Based on BERT
With the development of deep neural networks,more and more models have shifted from focusing on the literal meaning of natural language to focusing on the semantic information of natural language texts.In order to obtain the semantic infor-mation of the text,the model needs a large number of texts to be trained.In 2018,BERT is born,and a pre-training model for natu-ral language is proposed.The text similarity algorithm based on the BERT pre-training model has also received extensive attention.Since the attention mechanism of BERT does not pay attention to the timing information of the text,and the timing information of the text is an important feature in the text similarity comparison,this paper improves the BERT model for this part of the timing informa-tion thot BERT lacks.Finally,the validation and test sets of the LCQMC dataset are improved by 2.07%and 0.87%,respectively,reaching an accuracy of 89.39%and 87.41%,respectively.
deep neural networkBERTsemantic informationtiming information
万方数据
维普
