Generative Adversarial Networks with Position Encoding for Text Generation
For the problem that text logic is not logical due to the disorder of the position relationship between words when text is generated in the current adversarial network text generation model,we propose a text generation model(Position-Encoding GAN,PE_GAN)which introduces a position encoding mechanism to adversarial network,and discusses and validates it.By introducing positional encoding to the adversarial neural network model,the position relationship between words in the text is marked using word vectors with positional encoding.The generator and discriminator utilize the gate mechanism of the GRU neural network to alleviate gradient vanishing,while employing the Monte Carlo strategy to reduce the risk of overfitting and improve the accuracy of generated text.To verify the effectiveness of the PE_GAN,open source data and text from novels and news articles obtained through web scraping area used as the experimental dataset.It is showed that the difference in loss values between generator and discriminator in this model is smaller than that of the comparison models,indicating the generated text is closer to real text.In comparison to the Gumbel-softmax GAN,seq-GAN and LFMGAN,the PE_GAN shows a signficant improvement in BLEU-2,BLEU-3 and BLEU-4 values.This suggests that intro-ducing positional encoding mechanism can improve the logical coherence of the generated text,indicating that the model has good appli-cability.
国家科技期刊平台
NSTL
万方数据
