Graph Neural Networks(GNNs)have garnered increasing attention for their ability to model non-Euclidean graph structures and complex features.They have been applied extensively in various application domains,such as recommender systems,link prediction,and traffic prediction.However,training GNN models on large-scale data poses several challenges,such as irregular graph structures,complex node features,and dependent graph training samples.These challenges can put a strain on computation efficiency,memory management,and the communication cost of distributed computing.To overcome these challenges,many researchers have focused on optimi-zing application methods,algorithm models,programming frameworks,and hardware design.This survey specifically focuses on algorithm optimization and framework acceleration for large-scale GNN models.By examining related works in these areas,this survey aims to help readers understand the existing research as well as lay the foundation for co-optimizing GNN algorithms and frameworks for large-scale data.This survey is structured as follows.Firstly,we provide an overview of the challenges faced by GNNs in large-scale applications and the major optimization methods used to deal with these challenges.In addition,we compare our survey with existing surveys on GNNs.The major difference is that our survey focuses specifically on GNN models in large-scale applications.We summarize and analyze related works on GNN algorithms and framework optimization with a focus on scalability.In the second section,we provide a brief overview of the message passing mechanism and classify GNN models into four categories:Graph Convolutional Networks,Graph Attention Networks,Graph Recurrent Neural Networks,and Graph Autoencoder.For each category,we introduce the major network design,including propagation and aggregation strategies,and analyze the corresponding challenges of processing large-scale data.Furthermore,we provide a summary of the challenges faced by GNN models in large-scale applications,in terms of full-batch and mini-batch training modes.Thirdly,we classify and analyze GNN algorithms for large-scale data.We focus on sampling-based GNNs at different granularities,which use node-,layer-,and subgraph-based sampling strategies to optimize the mini-batch training of GNNs.Specifically,node-based sampling strategies usually select a fixed number of neighbors for each node,layer-based sampling methods operate at each GNN layer,and subgraph-based sampling approaches attempt to find dense subgraphs as mini batches.We provide a summary of each type of sampling strategy,including its key ideas,related works,and a discussion of its advantages and disadvantages.In the fourth section of this survey,we introduce mainstream programming frameworks for GNN models and related optimization techniques for framework acceleration.We briefly introduce mainstream programming frameworks one by one,such as DGL,PyG,Graph-Learn,and also summarize their characteristics.We divide these optimization strategies into five categories:data partition,task scheduling,parallel execution,memory management,and other methods.Finally,we summarize this survey.We also provide prospects for future work in optimizing GNN models and accelerating frameworks for large-scale data,such as reducing redundant computation,algorithm and framework co-optimization,graph-aware optimizations,support for complex graphs,flexible scheduling based on hardware features,optimizations on distributed platforms,framework and hardware co-optimization and minimizing node representation dimensions.
NETL
NSTL
万方数据
