首页|A disk I/O optimized system for concurrent graph processing jobs

A disk I/O optimized system for concurrent graph processing jobs

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
In order to analyze and process the large graphs with high cost efficiency,researchers have developed a number of out-of-core graph processing systems in recent years based on just one commodity computer.On the other hand,with the rapidly growing need of analyzing graphs in the real-world,graph processing systems have to efficiently handle massive concurrent graph processing(CGP)jobs.Unfortunately,due to the inherent design for single graph processing job,existing out-of-core graph processing systems usually incur unnecessary data accesses and severe competition of I/O bandwidth when handling the CGP jobs.In this paper,we propose GraphCP,a disk I/O optimized out-of-core graph processing system that efficiently supports the processing of CGP jobs.GraphCP proposes a benefit-aware sharing execution model to share the I/O access and processing of graph data among the CGP jobs and adaptively schedule the graph data loading based on the states of vertices,which efficiently overcomes above challenges faced by existing out-of-core graph processing systems.Moreover,GraphCP adopts a dependency-based future-vertex updating model so as to reduce disk I/Os in the future iterations.In addition,GraphCP organizes the graph data with a Source-Sorted Sub-Block graph representation for better processing capacity and I/O access locality.Extensive evaluation results show that GraphCP is 20.5× and 8.9× faster than two out-of-core graph processing systems GridGraph and GraphZ,and 3.5× and 1.7× faster than two state-of-art concurrent graph processing systems Seraph and GraphSO.

graph processingdisk I/Oconcurrent jobs

Xianghao XU、Fang WANG、Hong JIANG、Yongli CHENG、Dan FENG、Peng FANG

展开 >

School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China

Wuhan National Laboratory for Optoelectronics,Huazhong University of Science and Technology,Wuhan 430074,China

Department of Computer Science & Engineering,University of Texas at Arlington,Arlington,TX 76019,USA

College of Computer and Data Science,Fuzhou University,Fuzhou 350108,China

Zhejiang Lab,Hangzhou 311121,China

展开 >

National Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Defense Preliminary Research ProjectFundamental Research Funds for the Central Universities,the Open Project Program of Wuhan National Laboratory for OptoelectrNatural Science Foundation of Fujian ProvinceZhejiang provincial"Ten Thousand Talents Program"Centerinitiated Research Project of Zhejiang Lab

6183202061821003U1705261315110102022022WNLOKF0172020J014932021R520072021DA0AM01

2024

10.1007/s11704-023-2361-0

计算机科学前沿
高等教育出版社

计算机科学前沿

CSTPCDEI
影响因子：0.303
ISSN：2095-2228
年,卷(期)：2024.18(3)
  • 38