首页|A Pyramid Fusion MLP for Dense Prediction

A Pyramid Fusion MLP for Dense Prediction

扫码查看
原文链接
  • NETL
  • NSTL
  • IEEE
Recently, MLP-based architectures have achieved competitive performance with convolutional neural networks (CNNs) and vision transformers (ViTs) across various vision tasks. However, most MLP-based methods introduce local feature interactions to facilitate direct adaptation to downstream tasks, thereby lacking the ability to capture global visual dependencies and multi-scale context, ultimately resulting in unsatisfactory performance on dense prediction. This paper proposes a competitive and effective MLP-based architecture called Pyramid Fusion MLP (PFMLP) to address the above limitation. Specifically, each block in PFMLP introduces multi-scale pooling and fully connected layers to generate feature pyramids, which are subsequently fused using up-sample layers and an additional fully connected layer. Employing different down-sample rates allows us to obtain diverse receptive fields, enabling the model to simultaneously capture long-range dependencies and fine-grained cues, thereby exploiting the potential of global context information and enhancing the spatial representation power of the model. Our PFMLP is the first lightweight MLP to obtain comparable results with state-of-the-art CNNs and ViTs on the ImageNet-1K benchmark. With larger FLOPs, it exceeds state-of-the-art CNNs, ViTs, and MLPs under similar computational complexity. Furthermore, experiments in object detection, instance segmentation, and semantic segmentation demonstrate that the visual representation acquired from PFMLP can be seamlessly transferred to downstream tasks, producing competitive results. All materials contain the training codes and logs are released at https://github.com/huangqiuyu/PFMLP.

VisualizationTransformersComputer architectureConvolutionComputational modelingContext modelingAggregatesSemantic segmentationObject detectionKernel

Qiuyu Huang、Zequn Jie、Lin Ma、Li Shen、Shenqi Lai

展开 >

School of Cyber Science and Technology, Sun Yat-sen University, Shenzhen Campus, Shenzhen, China|Meituan, Beijing, China

Meituan, Beijing, China

School of Cyber Science and Technology, Sun Yat-sen University, Shenzhen Campus, Shenzhen, China

College of Computer Science and Technology, Zhejiang University, Hangzhou, China

展开 >

2025

10.1109/TIP.2025.3526054

IEEE transactions on image processing

IEEE transactions on image processing

ISSN：
年,卷(期)：2025.34(1)
  • 83