Traditional MPI(Message Passing Interface)computing is characterized by the migration of data to computing,which has inherent shortcomings for computing tasks with alarge amount of data.This paper proposes a distributed file system architecture supporting MPI,the architecture and implementation of MPIFS.The file system is based on HDFS(Hadoop Distributed File System),enabling MPI to support both computation-intensive and data-intensive computations on MPIFS.In this paper,two types of batch word frequency statistics experiments are set up.All the data required for the experiments are distributed and stored in the MPIFS distributed file system,and the data access is achieved by calling the unified data interface provided by the system.Through experiments,a single computing node locally calculates m size files in the same time as n nodes parallelly calculates m X n size files,and the total amount of files in MPIFS remains unchanged.The number of computing nodes decreases and the computing time t becomes longer.It can be concluded that the MPIFS file system architecture is feasible and can support MPI to realize the parallel computation of computation to data migration.
NETL
NSTL
万方数据
维普
