In recent years,network anomaly detection model based on deep learning has become a research hotspot in the area,getting outstanding achievements in experimental environments.Howev-er,there is a lack of research related with the generalization ability of those models.The paper con-structed three representative network anomaly detection models based on multi-layer perceptron,1-D convolutional neural network and deep auto-encoder,and trained on CICIDS2017 and CICIDS2018.Then,the evaluation experiments are carried out in a cross way to quantify its generalization ability.The experimental results show that the accuracy of the models has declined by 20.78%,23.18%and 11.13%on average,which proves that the generalization performance of the deep network anomaly detection model is a serious problem,and reveals the pitfall of applying deep learning technology to network security and the key obstacle to its practical deployment.Finally,the summary and analysis of this problem is discussed and the potential solutions are put forward.