首页|M2Trans: Multi-Modal Regularized Coarse-to-Fine Transformer for Ultrasound Image Super-Resolution

M2Trans: Multi-Modal Regularized Coarse-to-Fine Transformer for Ultrasound Image Super-Resolution

扫码查看
原文链接
  • NETL
  • NSTL
  • IEEE
Ultrasound image super-resolution (SR) aims to transform low-resolution images into high-resolution ones, thereby restoring intricate details crucial for improved diagnostic accuracy. However, prevailing methods relying solely on image modality guidance and pixel-wise loss functions struggle to capture the distinct characteristics of medical images, such as unique texture patterns and specific colors harboring critical diagnostic information. To overcome these challenges, this paper introduces the Multi-Modal Regularized Coarse-to-fine Transformer (M2Trans) for Ultrasound Image SR. By integrating the text modality, we establish joint image-text guidance during training, leveraging the medical CLIP model to incorporate richer priors from text descriptions into the SR optimization process, enhancing detail, structure, and semantic recovery. Furthermore, we propose a novel coarse-to-fine transformer comprising multiple branches infused with self-attention and frequency transforms to efficiently capture signal dependencies across different scales. Extensive experimental results demonstrate significant improvements over state-of-the-art methods on benchmark datasets, including CCA-US, US-CASE, and our newly created dataset MMUS1K, with a minimum improvement of 0.17dB, 0.30dB, and 0.28dB in terms of PSNR.

Ultrasonic imagingTransformersMedical diagnostic imagingSuperresolutionTask analysisSemanticsComputational modeling

Zhangkai Ni、Runyu Xiao、Wenhan Yang、Hanli Wang、Zhihua Wang、Lihua Xiang、Liping Sun

展开 >

Department of Computer Science and Technology and the Key Laboratory of Embedded System and Service Computing (Ministry of Education), Tongji University, Shanghai, China

Pengcheng Laboratory, Shenzhen, China

Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU-BIT University, Shenzhen, China

Department of Medical Ultrasound, Center of Tumor Minimally Invasive Treatment, Shanghai Tenth People's Hospital, Ultrasound Research and Education Institute, Clinical Research Center for Interventional Medicine, School of Medicine, Tongji University, Shanghai, China|Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China

展开 >

2025

10.1109/JBHI.2024.3454068

IEEE journal of biomedical and health informatics

IEEE journal of biomedical and health informatics

ISSN：
年,卷(期)：2025.29(5)
  • 50