Survey of Distributed Computing Frameworks for Supporting Big Data Analysis

扫码查看

原文链接

万方数据
维普

外文摘要：Distributed computing frameworks are the fundamental component of distributed computing systems.They provide an essential way to support the efficient processing of big data on clusters or cloud.The size of big data increases at a pace that is faster than the increase in the big data processing capacity of clusters.Thus,distributed computing frameworks based on the MapReduce computing model are not adequate to support big data analysis tasks which often require running complex analytical algorithms on extremely big data sets in terabytes.In performing such tasks,these frameworks face three challenges:computational inefficiency due to high I/O and communication costs,non-scalability to big data due to memory limit,and limited analytical algorithms because many serial algorithms cannot be implemented in the MapReduce programming model.New distributed computing frameworks need to be developed to conquer these challenges.In this paper,we review MapReduce-type distributed computing frameworks that are currently used in handling big data and discuss their problems when conducting big data analysis.In addition,we present a non-MapReduce distributed computing framework that has the potential to overcome big data analysis challenges.

外文关键词：

distributed computing frameworksbig data analysisapproximate computingMapReduce computing model

作者：

Xudong Sun、Yulin He、Dingming Wu、Joshua Zhexue Huang

展开 >

作者单位：

College of Computer Science and Software Engineering,Shenzhen University,Shenzhen 518060,China

Guangdong Laboratory of Artificial Intelligence and Digital Economy(SZ),Shenzhen 518107,China

基金：

National Natural Science Foundation of ChinaBasic Research Foundations of ShenzhenBasic Research Foundations of Shenzhen

项目编号：

61972261JCYJ 20210324093609026JCYJ20200813091134001

出版年：

2023

DOI：

10.26599/BDMA.2022.9020014

大数据挖掘与分析(英文版)

CSCDEI

ISSN：

年,卷(期)：2023.6(2)

参考文献量3