首页|利用跨模态轻量级YOLOv5模型的PET/CT肺部肿瘤检测

利用跨模态轻量级YOLOv5模型的PET/CT肺部肿瘤检测

扫码查看
多模态医学图像可在同一病灶处提供更多语义信息,针对跨模态语义相关性未充分考虑和模型复杂度过高的问题,该文提出基于跨模态轻量级YOLOv5(CL-YOLOv5)的肺部肿瘤检测模型.首先,提出学习正电子发射型断层显像(PET)、计算机断层扫描(CT)和PET/CT不同模态语义信息的3分支网络;然后,设计跨模态交互式增强块充分学习多模态语义相关性,余弦重加权计算Transformer高效学习全局特征关系,交互式增强网络提取病灶的能力;最后,提出双分支轻量块,激活函数簇(ACON)瓶颈结构降低参数同时增加网络深度和鲁棒性,另一分支为密集连接的递进重参卷积,特征传递达到最大化,递进空间交互高效地学习多模态特征.在肺部肿瘤PET/CT多模态数据集中,该文模型获得94.76%mAP最优性能和3238 s最高效率,以及0.81 M参数量,较YOLOv5s和EfficientDet-d0降低7.7倍和5.3倍,多模态对比实验中总体上优于现有的先进方法,消融实验和热力图可视化进一步验证.
CL-YOLOv5: PET/CT Lung Cancer Detection With Cross-modal Light-weight YOLOv5 Model
Multimodal medical images can provide more semantic information at the same lesion. To address the problems that cross-modal semantic features are not fully considered and model complexity is too high, a Cross-modal Lightweight YOLOv5(CL-YOLOv5) lung cancer detection model is proposed. Firstly, three-branch network is proposed to learn semantic information of Positron Emission Tomography (PET), Computed Tomography (CT) and PET/CT; Secondly, Cross-modal Interactive Enhancement block is designed to fully learn multimodal semantic correlation, cosine reweighted Transformer efficiently learns global feature relationship, interactive enhancement network extracts lesion features; Finally, dual-branch lightweight block is proposed, ACtivate Or Not (ACON) bottleneck structure reduces parameters while increasing network depth and robustness, the other branch is densely connected recursive re-parametric convolution with maximized feature transfer, recursive spatial interaction efficiently learning multimodal features. In lung cancer PET/CT multimodal dataset, the model in this paper achieves 94.76% mAP optimal performance and 3238 s highest efficiency, 0.81 M parameters are obtained, which is 7.7 times and 5.3 times lower than YOLOv5s and EfficientDet-d0, overall outperforms existing state-of-the-art methods in multimodal comparative experiments.In multi-modal comparison experiment, it is generally better than the existing advanced methods, further verification by ablation experiments and heat map visualization ablation experiment.

YOLOv5Cross-modal interactive enhancement blockDual-branch lightweight blockPositron Emission Tomography / Computed Tomography (PET/CT) multi-modal lung cancer image

周涛、叶鑫宇、刘凤珍、陆惠玲

展开 >

北方民族大学计算机科学与工程学院 银川 750021

北方民族大学图像图形智能处理国家民委重点实验室 银川 750021

宁夏医科大学医学信息工程学院 银川 750004

YOLOv5 跨模态交互式增强块 双分支轻量块 PET/CT多模态肺部肿瘤影像

国家自然科学基金宁夏回族自治区自然科学基金宁夏回族自治区重点研发计划

620620032022AAC031492020BEB04022

2024

电子与信息学报
中国科学院电子学研究所 国家自然科学基金委员会信息科学部

电子与信息学报

CSTPCD北大核心
影响因子:1.302
ISSN:1009-5896
年,卷(期):2024.46(2)
  • 20