From graph convolution networks to graph scattering networks:a survey
In image processing and computer graphics,non-Euclidean data such as graphs have gained increasing attention in recent years because Euclidean data such as images and videos fail to represent data with structure information.Com-pared with traditional Euclidean data,the scale of a graph can be arbitrary large.The structure of a graph usually contains information such as the relation between vertices,the logical sequences,and the properties of graph itself.While images can be easily converted into graphs based on the Euclidean position of pixels,graphs(especially for irregular graphs)can merely be converted into images.Therefore,graphs require a higher level of representation learning compared with tradi-tional Euclidean data.However,in the era of deep learning,traditional convolution neural networks(CNNs)fail to learn representations for graphs due to permutation covariance for nodewise features and permutation invariance for outputs such as classification labels.The performance of CNNs in graph representation learning is still limited even if inputs are aug-mented by arbitrary permutation during training to learn permutation covariance.The development of graph neural networks and graph convolution operations achieves milestone in representation learning of non-Euclidean data such as graphs.Com-monly,graph convolution neural networks(GCNs)can be divided into two categories:spatial GCNs and spectral GCNs.Spatial GCNs focus on the establishment of neighborhood and update with aggregation functions that combine the features of the center vertex and its neighbors.Though GCNs based on neighborhood feature aggregation encourage the propagation of nodewise features,deep GCNs usually suffer from oversmoothness issue,and the features of vertices become indistinguish-able.Therefore,later works consider introducing skip connections in deep GCNs or constructing shallow GCNs with multi-scale neighborhood considered within each convolution to alleviate this issue.Spectral GCNs focus on the graph spectral theorem and update their parameters by signal filtering in spectral domain with designed filters.However,eigen-decomposition of graph shift operator is costly for large graphs because its computation complexity is O(N3).Therefore,spectral GCNs usually apply K-order polynomials(i.e.,Chebyshev polynomials)to approximate the target filters and avoid eigen-decomposition.Though spectral GCNs may avoid oversmoothness issue with graph filter design,the limited number of learnable parameters and filter responses of spectral GCNs usually limit their expression ability.Spatial GCNs and spectral GCNs are not necessarily independent from one another.For example,1-order Chebyshev polynomials with diffusion matrix are equivalent to feature aggregation within 1-hop neighborhood.Therefore,spatial GCNs based on feature aggregation with diffusion Laplacian matrix or lazy random walk matrix usually have the spectral form,which bridges the spatial GCNs and spectral GCNs.The rapid development in graph representation learning gives rise to the demand for sur-vey and review that summarize existing works and serve as guidance for beginners.Currently,graph neural networks such as graph convolution neural networks,graph embeddings,and graph autoencoders have been reviewed.However,current surveys and reviews lack one domain in graph representation learning:graph scattering transforms(GSTs)and graph scat-tering networks(GSNs).GSNs are non-trainable spectral GCNs based on wavelet decomposition.With the benefit of multi-scale wavelets and the structure of networks,GSNs generate diverse features with nearly nonoverlapping frequency responses in the spectral domain.As one of the newly developed graph representation learning methods,GSNs are used in tasks such as graph classification and node classification.Recent works employed graph scattering transform to spatial GCNs to overcome the oversmoothness issue.Compared with spectral GCNs,GSNs generate diverse features that strengthen the expressive capability of model without introducing the oversmoothness issue.However,the nontrainable property of GSNs may limit the flexibility in graph representation learning on different graph datasets with different distribu-tions of spectrum.GSNs suffer from the exponential growth of diffusion paths with the increase of scattering layers,which limit the depth of GSNs in practice.In this paper,a survey comprehensively reviews the designs from GCNs to GSNs.First,GCNs are divided into two categories:spatial GCNs and spectral GCNs.Spatial GCNs are categorized into the follow-ing types:1)diffusion-based GCNs,2)GCNs on large graphs with neighbor sampling or subgraph sampling,3)GCNs with attention mechanism,and 4)GCNs with dynamic neighborhood construction.Spectral GCNs are reviewed according to different filters(filter kernels):Chebyshev polynomials,Cayley polynomials,and K-order polynomials.After address-ing the drawbacks of spatial GCNs and spectral GCNs,the definition of GSTs,the structure of classical GSNs,and the advantages of GSNs compared with GCNs are introduced.The current arts of GSNs are elaborated from the perspectives of network design along with application and stability in theory.The networks and application of graph scattering transform are reviewed in the following categories:1)classical GSNs,2)graph scattering transforms in GCNs to solve the over-smoothness issue,3)graph attention networks with GSTs,4)graph scattering transform on spatial-temporal graphs,5)reducing scattering paths of GSNs via pruning to increase the efficiency of graph scattering transform,6)GSNs with mul-tiresolution graphs,and 7)trainable GSNs with wavelet scale selection and learnable spectral filters.In theory,the frame theorem and the stability theorem under signal and topology perturbation,respectively,are concluded.The limitations of GSNs(GSTs)in current works are analyzed,and possible directions for the development of graph scattering technics in the future are proposed.
deep learninggraph convolution network(GCN)graph scattering network(GSN)representation learningstabilitysignal perturbationtopology perturbation
NETL
NSTL
万方数据
