Machine Reading Comprehension Model Based on MacBERT and Adversarial Training
Machine reading comprehension is designed to allow machines to understand natural language texts,resembling humans,and perform question-answering tasks accordingly.In recent years,owing to the development of deep learning and large-scale datasets,machine reading comprehension has received widespread attention.However,input problems in practical applications typically involve various noises and interferences,which affect the prediction results of a model.To improve the generalizability and robustness of a model,a machine reading comprehension model based on Masked language modeling as correction Bidirectional Encoder Representations from Transformers(MacBERT)and Adversarial Training(AT)is proposed.First,MacBERT is used to convert input questions and texts into word embeddings and vector representations.Subsequently,a small perturbation is added to the original word vector based on the gradient change of the original sample backpropagation to generate an adversarial sample.Finally,the original and adversarial samples are input into a Bidirectional Long Short-Term Memory(BiLSTM)network to further extract the contextual features of the text and output the predicted answer.Experimental results show that the F1 and Exact Matching(EM)values of this model on the simplified Chinese dataset CMRC2018 improve by 1.39 and 3.85 percentage points,respectively,compared with those of the baseline model.Meanwhile,the F1 and EM values on the traditional Chinese dataset DRCD improve by 1.22 and 1.71 percentage points,respectively,compared with those of the baseline model.Moreover,the F1 and EM values on the English dataset SQuADv1.1 improve by 2.86 and 1.85 percentage points,respectively,compared with those of the baseline model.The experimental results are better than those of most existing machine reading comprehension models.Based on actual question-answering results,the proposed model outperforms the baseline model in terms of robustness and generalizability;additionally,it performs better when the input problems contain noise.
machine reading comprehensionAdversarial Training(AT)pre-trained modelMasked language
国家科技期刊平台
NSTL
万方数据
