首页|Deformable Convolution-Enhanced Hierarchical Transformer With Spectral-Spatial Cluster Attention for Hyperspectral Image Classification

Deformable Convolution-Enhanced Hierarchical Transformer With Spectral-Spatial Cluster Attention for Hyperspectral Image Classification

扫码查看
原文链接
  • NETL
  • NSTL
  • IEEE
Vision Transformer (ViT), known for capturing non-local features, is an effective tool for hyperspectral image classification (HSIC). However, ViT’s multi-head self-attention (MHSA) mechanism often struggles to balance local details and long-range relationships for complex high-dimensional data, leading to a loss in spectral-spatial information representation. To address this issue, we propose a deformable convolution-enhanced hierarchical Transformer with spectral-spatial cluster attention (SClusterFormer) for HSIC. The model incorporates a unique cluster attention mechanism that utilizes spectral angle similarity and Euclidean distance metrics to enhance the representation of fine-grained homogenous local details and improve discrimination of non-local structures in 3D HSI and 2D morphological data, respectively. Additionally, a dual-branch multiscale deformable convolution framework augmented with frequency-based spectral attention is designed to capture both the discrepancy patterns in high-frequency and overall trend of the spectral profile in low-frequency. Finally, we utilize a cross-feature pixel-level fusion module for collaborative cross-learning and fusion of the results from the dual-branch framework. Comprehensive experiments conducted on multiple HSIC datasets validate the superiority of our proposed SClusterFormer model, which outperforms existing methods. The source code of SClusterFormer is available at https://github.com/Fang666666/HSIC_SClusterFormer.

Feature extractionTransformersConvolutionComputational modelingData miningSunMixersComputer scienceSupport vector machinesRedundancy

Yu Fang、Le Sun、Yuhui Zheng、Zebin Wu

展开 >

School of Computer Science, Nanjing University of Information Science and Technology, Nanjing, China|School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

School of Computer Science and Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing, China

School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

2025

10.1109/TIP.2024.3522809

IEEE transactions on image processing

IEEE transactions on image processing

ISSN：
年,卷(期)：2025.34(1)
  • 37