Design and Implementation of a High-Performance Cloud-Native Big Data Platform
After more than ten years,the traditional big data platform software has become increasingly mature.In recent years,the cloud-na-tive architecture featuring containerization has become the preferred solution for infrastructure construction.Especially under the trend ofinte-grating high-performance computing technology into the cloud-native environment,the design of a new generation of big data platforms is fac-ing different challenges.These challenges involve job scheduling in a cloud-native environment,containerized adaptation of high-perfor-mance networks,and storage management under a storage-compute separation architecture.In response to these problems,this paper propos-es a set of key technologies for a high-performance cloud-native big data platform,including the multi-mode workload containerized schedul-ing technology,the containerized RDMA data exchange technology,and the cloud-native storage-compute separation architecture.And on this basis,developed the OMBD big data platform.OMBD can effectively adapt to the characteristics of high-performance cloud native envi-ronment,and realize effective scheduling and efficient execution of multi-mode big data jobs in containerized clusters with high-performance network cards with systematic technical solutions.Experimental data and real-world application results prove that OMBD is a practical and effi-cient production-grade big data platform.
big datacloud-nativehigh-performance computingcontainerizationplatform software
万方数据
维普
