Research on Load Balancing Method Based on Node Characteristics in Heterogeneous Hadoop Clusters
To solve the load balancing problem of each node in heterogeneous Hadoop clusters,a node feature-based load balancing method is proposed.This method first analyzes the main indicators that determine the node per-formance.Then,according to the contribution of these indicators,the corresponding objective function is defined to de-scribe the node performance;At the same time,the relative load value of each node in the cluster is calculated by combining the node performance with the node busy state.The recommended value of storage utilization of each node is calculated by analyzing the relative load value of each node and the cluster storage utilization.Finally,the balance threshold is dynamically generated based on the current disk load status of the cluster.The experimental results show that the load balancing method proposed in this paper can provide a more reasonable balancing scheme for heterogene-ous Hadoop clusters,and can improve the running efficiency and shorten the execution time.
NETL
NSTL
万方数据
