DYNAMIC BALANCING PARTITION STRATEGY FOR LOAD BALANCING
In view of the unbalanced load on the reduce side caused by MapReduce computing framework processing inclined data sets,this paper proposes a dynamic balanced partition strategy.In the mapper stage,the data segmentation principle based on divide and conquer method was proposed to process the incoming data groups.The dynamic allocation principle was designed combined with the idea of the best adaptive algorithm to gradually allocate the segmented data blocks to the server.According to the partition index,it was allocated to each reduce node to achieve load balancing.The experimental results show that,compared with the two benchmark models,the average task execution time of the dynamic balanced partition strategy is reduced by 7.7％,which indicates that the dynamic balanced partition strategy can better solve the problem of data skew,reduce the task execution time,and verify the effectiveness of the model.
国家科技期刊平台
NSTL
万方数据
