Research on application attribute aware Yarn resource scheduling model
Hadoop applications have computation intensive and scheduling time attributes.However,the three built-in resource schedulers of the second generation resource manager,which is integrated in Hadoop big data platform,are unable to evenly distribute applications with the same attributes to the computing nodes,resulting in excessive load on some nodes and serious long tail effect of computing tasks.This paper presents an application attribute aware yarn load balancing scheduling model-APB Scheduler.APB Schedu-ler automatically perceive the application attributes,evenly allocate the containers of the same attribute ap-plication to the cluster computing nodes according to the dynamic resource plan,and use the NSGA-Ⅲ al-gorithm to complete the calculation of the optimal allocation scheme.Through experimental verification,APB Scheduler solves the container allocation skew problem of applications with the same content,and greatly improves the performance and stability of the cluster.
NETL
NSTL
万方数据
