Prediction Algorithm for Job Execution Time and Memory Based on Feature Similarity
Accurately predicting the required execution time and memory for a job is the key to improving the per-formance of job scheduling systems.However,the accuracy of the estimated execution time and memory by most users is poor.This paper proposes a prediction algorithm,LSH-Sim,based on the similarity of job features,which combines similar search and machine learning.Similar jobs in the historical job set were searched based on textual and numeri-cal features,and machine learning or the mean method was used to make predictions in the similar job set.Search for similar jobs with the locally sensitive hashing algorithm to shorten the prediction time while improving the prediction accuracy.Experiments were conducted using the historical job set from theNational Supercomputing Kunshan Center,Hefei Advanced Computing Center and Wuzhen Light Supercomputing Center.The experimental results show that compared to the simple prediction and improved templateprediction algorithms,LSH-Sim algorithm has a lower mean absolute error and a shorter prediction time.
NETL
NSTL
万方数据
