首页|面向算力网络的跨域数据管理方法

面向算力网络的跨域数据管理方法

扫码查看
跨域算力网络希望整合多个算力中心的计算和数据资源,但现有的方案对跨域文件和数据管理关注不够.提出了一种轻量级的跨域算力网络数据管理方案:通过文件系统协议转换,接入远程算力中心的并行文件系统存储资源;算力中心内部的存储资源作为一种补充,应对高IOPS应用;通过容器绑定技术,将远程存储挂载并绑定到指定目录.基于该方案的原型系统已经在高校校级计算平台部署运行.实测数据和用户体验显示,该方案能够满足常见高性能计算应用需求.
Cross-domain data management for computing power networks
Cross-domain computing power networks wish to integrate computational and data resources from multiple computing centers,but existing methods do not pay enough attention to cross-domain file and data management.In this paper,a lightweight data access scheme for cross-domain computing power networks was proposed:(1)accessing parallel file system storage resources of remote computing centers through file system protocol conversion;(2)local caching as a supplement to cope with high IOPS applications;and(3)mounting remote or local storage to specified directories through container binding technology.The prototype system based on this scheme had been deployed on high-performance computing centers in multiple universities.The measured data and user experience showed that the scheme in this paper could meet the requirements of common high-performance computing applications.

computing power networkparallel file systemdata managementheterogeneous storage resource

鲁蔚征、戴奇志、张策

展开 >

中国人民大学大型科学仪器共享平台,北京 100083

联旌智能科技(上海)有限公司,上海 200051

华中科技大学网络与计算中心,湖北 武汉 430074

算力网络 并行文件系统 数据管理 异构存储资源

国家重点研发计划

2020YFB1710004

2024

大数据
人民邮电出版社

大数据

CSTPCD
ISSN:2096-0271
年,卷(期):2024.10(2)
  • 22