A Method for Hyper-parameter Tuning of Neural Network Based on m×2 Regularized Cross-validation
Hyper-parameter tuning is a key issue in neural network modeling.From the viewpoint of the problems of traditional hyper-parameter tuning methods,we propose a hyper-parameter tuning method based on m×2 regularized cross-validation.The goal is to present a robust hyper-parameter tuning method with low computational cost suitable for complex models and large datasets.The idea of the proposed method is to select a small number of data from the complete dataset for tuning,so as to avoid the time-consuming problem of hyper-parameter tuning when the dataset is large.Then,on the basis of m×2 cross-validation,regularization is adopted to balance the distribution difference between the training set and the validation set to reduce the performance fluctuation caused by the distribution in-consistency.The signal-to-noise ratio is used as the metric of hyper-parameter tuning,so that the mean and variance of the model per-formance can be comprehensively considered.The orthogonal design is used to select a combination of hyper-parameters with low correlation to improve the tuning efficiency.The experimental results on the CoNLL 2003 dataset show that the proposed method can obtain a combination of hyper-parameters that is not significantly different from the grid search,and the tuning time can be significantly reduced by about 66％.
万方数据
维普
