气象大数据云平台仿真环境容器调度性能优化研究
Research on Optimization of Docker Scheduling Performance for Simulation Environment of Meteorological Big Data Cloud Platform
吴鹏 1韩同欣 1陈士旺 1聂元丁 1郑晓志2
作者信息
- 1. 国家气象信息中心,北京 100081
- 2. 广州市海珠区气象局,广州 510240
- 折叠
摘要
为实现2025年气象关键核心技术自主可控的目标,气象大数据云平台(简称天擎)建立了基于海光X86服务器和麒麟操作系统的仿真环境.在仿真平台运行中发现,基于容器技术的产品加工与流水线子系统容器调度性能较差,不能满足用户融入算法的时效要求.针对此问题,本文采用对比分析法,选取天擎仿真环境和业务环境的3种CPU芯片服务器和3种操作系统为研究对象,设计了一系列组合对比测试用例,找到了影响容器调度性能的关键因素—操作系统内核,并进一步分析了操作系统内核设置对系统实时性和吞吐量的影响以及适用的业务场景.最后给出了麒麟操作系统内核调整方法,通过调整内核设置,容器调度性能大幅提高,满足了产品加工系统的时效要求,为实现天擎的关键核心技术自主可控奠定基础.
Abstract
In order to achieve the goal of independent and controllable key core technologies for Meteo by 2025,the Meteo Big Data Cloud Platform(referred to as Tianqing)establishes a simulation environment based on Hygon X86 CPU and Kylin OS.However,in the operation of simulation platforms,it finds that the docker scheduling performance of data processing and assembly line subsystems based on Kubernetes is poor,which cannot meet the timeliness requirements of user integration algorithms.In response to this issue,this article adopts a comparative analysis method,selecting servers based on three types of CPU and three types of operating systems from the simulation environment and business environment for Tianqing as the research objects.A series of combined comparative test cases are designed.It finds that the kernel is the key factor affecting docker scheduling performance.Further analysis is conducted on the impact of operating system kernel settings on real-time and throughput,as well as the suitable business scenarios.Finally,a method for adjusting the Kylin OS kernel is provided.By adjusting the kernel settings,the docker scheduling performance significantly improves,meeting the timeliness requirements of the data processing system and laying the foundation for achieving self-supporting of the key core technology of Tianqing.
关键词
气象大数据云平台/自主可控/麒麟操作系统/容器调度Key words
Meteo Big Data Cloud Platform/self-supporting/Kylin OS/docker scheduling引用本文复制引用
基金项目
国家气象信息中心信息网络安全与"信创"技术研发创新团队攻关任务项目(NMIC-202011-05)
广东省气象局科学技术研究项目(GRMC2022Z05)
广东省气象局科学技术研究项目(GRMC2021XQ03)
出版年
2024
NETL
NSTL
万方数据