Atmospheric Temperature Prediction Based on MLP and Transformer Models
Based on the atmospheric temperature data from January 1,2015 to December 21,2020 in Yuncheng City,the Multilayer Perceptron(MLP)model and Transformer model are used to forecast the atmos-pheric temperature in Yuncheng City in this paper.Since weather data is highly time-sequential,we used two-layer and four-layer versions of the Transformer and MLP models to conduct multiple comparative experi-ments over 3-day,5-day,and 7-day periods.The results showed that the 7-day mean squared error(MSE)of the MLP-4 model was 3.2649,and the 3-day MSE of the Transformer-4 model was 5.3767,both demonstrating relatively high prediction accuracy.Moreover,the MLP model's accuracy in predicting tempera-ture was higher than that of the Transformer model.The MLP-2 model's MSEs were 3.2662,3.2996,and 3.3579,while the MLP-4 model's MSEs were 3.2674,3.2996,and 3.2649,showing minor fluctuations but re-maining stable.In contrast,the MSEs of the Transformer-2 model were 5.6225,5.9491,and 5.3892,and the Transformer-4 model's MSEs were 5.3767,6.3787,and 6.1108.Increasing the model layers and parame-ter count led to larger MSEs and overfitting.The overfitting phenomenon seen in the Transformer model was due to its excessive size(close to 4 million parameters),while the study datas only included 1,531 sets.Despite u-sing regularization methods like weight decay and dropout,the model still overfitted the 1,531 data sets,resul-ting in a decline in prediction accuracy.
temperature predictionMLP modelTransformer modelneural networks
NETL
NSTL
万方数据
