System performance optimization practice for big data scenarios
In the existing large-scale distributed environments, there is still much room for improvement in the performance and computational efficiency of big data applications. However, performance analysis and optimization in large-scale environments requires a large number of human resources from domain experts. This paper proposes a general low-performance query statement detection and optimization process for performance optimization in big data applications, summarizes four types of low-performance behaviors that significantly affect the performance of big data applications, and proposes specific optimization strategies for each type of low-performance behavior. Finally, through experimental evaluation, the effectiveness of the optimization scheme in actual large-scale cluster is verified.
Hadoopbig data systemperformance optimizationtuning tool
万方数据
维普
