仪表技术2024,Issue(2) :1-5,38.

基于HBase的极向场电源系统异构数据存储设计

Design of Heterogeneous Data Storage for Poloidal Field Power Supply System Based on HBase

许昌涛 黄连生
仪表技术2024,Issue(2) :1-5,38.

基于HBase的极向场电源系统异构数据存储设计

Design of Heterogeneous Data Storage for Poloidal Field Power Supply System Based on HBase

许昌涛 1黄连生1
扫码查看

作者信息

  • 1. 中国科学院 合肥物质科学研究院,安徽 合肥 230031;中国科学技术大学,安徽 合肥 230026
  • 折叠

摘要

极向场电源系统是全超导托卡马克核聚变实验(EAST)装置的重要组成部分.EAST装置的运行和电源系统的日常运行与维护不仅产生海量结构化时序数据,还生成文档、表格、音频和视频等非结构化数据文件.传统的关系型数据库和聚变领域MDSplus数据库不能满足异构数据的统一存储管理.搭建分布式集群,选取分布式数据库HBase,提出基于时间序列的行键设计来实现数据均衡分布和并行访问节点提高读取效率,提出基于降序最佳适应算法的文件合并策略设计来实现小文件的高效合并,充分利用数据块存储空间.实验结果表明,该种异构数据存储设计方案满足了存储需求.

Abstract

The polar field power supply system is an important component of the EAST device.The operation of the EAST device and the daily operation and maintenance of the power system can not only generate massive structured tem-poral data,but also produce unstructured data files such as documents,tables,audio,and video.Since traditional rela-tional databases and MDSplus databases in the fusion field cannot meet the unified storage and management of heteroge-neous data,this paper build a distributed cluster,and select Distributed database HBase.A row key design based on time series is proposed to achieve balanced data distribution and improve the reading efficiency of parallel access nodes.A file consolidation strategy design based on BFD algorithm is also proposed to achieve efficient consolidation of small files,and make full use of data block storage space.The experimental results indicate that this heterogeneous data storage design scheme meets the storage requirements.

关键词

极向场电源/分布式存储/行键设计/文件合并

Key words

poloidal power supply/distributed storage/row key design/file compacting

引用本文复制引用

基金项目

中国科学院重大科技基础设施维修改造项目(DSS-WXGZ-2021-0009)

安徽省重大专项(202003a05020019)

中科院等离子体物理研究所拔尖基金(DSJJ-2021-07)

出版年

2024
仪表技术
上海市仪器仪表学会,上海仪器仪表研究所等

仪表技术

影响因子:0.217
ISSN:1006-2394
参考文献量10
段落导航相关论文