With the continuous development of financial markets and changes in the global economy,accurately predicting stock market indices has become a key focus for investors and decision-makers.This study aims to explore the application of the Trans-former model and its attention mechanism in the prediction of financial indices within deep learning neural networks.By discard-ing conventional control variable designs and adopting a high-order autoregressive model based on historical stock index data,this paper innovatively proposes three Transformer model variants:Multi-attention Transformer,GRU Transformer,and Attention-Free Transformer,and compares their performance under both single-step iterative prediction and multi-step prediction methods.Empirical analysis is based on the daily data of the Shanghai Stock Exchange Index from January 1,2000,to March 11,2024.The data was expanded and standardized using Python.The results show that the GRU Transformer model combined with single-step iterative prediction has the lowest mean squared error(MSE)on the test set,at 0.00041,and performs excellently in terms of parameter efficiency and runtime,indicating significant advantages in prediction accuracy,parameter efficiency,and runtime.The innovations of this paper include simplifying the model structure while maintaining prediction accuracy by using a high-order au-toregressive model based on historical time series data;proposing and validating the effectiveness of three Transformer model variants in financial time series prediction;and comparing the effects of combining single-step iterative prediction and multi-step prediction methods.This study provides new perspectives and methods for financial market analysis and prediction,and future re-search can further validate the model's effectiveness and explore other potential improvement strategies.
Higher-order Autoregressive ModelTransformer ModelAttention MechanismFinancial Index Prediction
万方数据
