面向复杂场景的大数据应用调度方法研究
Research on Big Data Application Scheduling Method for Complex Scenarios
厉承轩 1杨美红 2郭莹 2张虎 2孙明辉1
作者信息
- 1. 齐鲁工业大学(山东省科学院)山东省计算中心 济南 250000;山东省计算中心(国家超级计算济南中心) 济南 250000
- 2. 山东省计算中心(国家超级计算济南中心) 济南 250000
- 折叠
摘要
针对Kubernetes默认调度方法在调度复杂场景的大数据应用时,没有根据单个大数据服务的特性权衡影响其性能的因素重要性,从而严重影响了复杂大数据应用的运行性能.提出了基于层次结构分析法的大数据服务容器集群调度方法.该方法针对特定大数据服务的实际需求,给出大数据服务的性能影响因素.对大数据服务建立层次结构模型,构建对比矩阵,计算出最底层宿主机相对于最高层目标来说的权重序列.最终将该容器集群调度到权重最高的宿主机上.调度实验中,论文调度方法调度的服务上传数据用时更短、效果稳定性更高,能够有效提高Kubernetes之上大数据服务的处理效率,进而提高复杂大数据应用的性能.
Abstract
The Kubernetes default scheduling method fails to balance the importance of factors affecting the performance of a single big data service according to its characteristics when scheduling big data applications in complex scenarios,which seriously affects the performance of complex big data applications.This paper proposes a scheduling method of big data service container clus-ter based on hierarchy analysis.Based on the actual demand of specific big data services,this method gives the performance influ-encing factors of big data services.The hierarchical structure model of big data services is established,and the pairwise comparison matrix is constructed to calculate the weight sequence of the lowest level host relative to the highest level target.Finally,the contain-er cluster is scheduled to the host with the highest weight.In the scheduling experiment,the service scheduled by the scheduling method in this paper takes less time to upload data and has higher effect stability,which can effectively improve the processing effi-ciency of big data service on Kubernetes,and then improve the performance of complex big data application.
关键词
Docker/Kubernetes/调度/大数据/微服务Key words
Docker/Kubernetes/scheduling/big data/microservice引用本文复制引用
基金项目
山东省计算中心(国家超级计算济南中心)项目(2020KJC-ZD01)
出版年
2024
国家科技期刊平台
NSTL
万方数据