基于混合知识蒸馏的轻量级胸部疾病分类算法
Lightweight thoracic disease classification algorithm based on mixed knowledge distillation
赖裕 1李锵 1聂为之 2白云鹏 3赵丰3
作者信息
- 1. 天津大学微电子学院,天津 300072
- 2. 天津大学电气自动化与信息工程学院,天津 300072
- 3. 天津市胸科医院心血管外科,天津 300222
- 折叠
摘要
针对现有胸部疾病分类算法参数量较大、对运行设备的硬件资源要求较高的问题,本文基于混合知识蒸馏(knowledge distillation,KD)的训练策略提出一种轻量级胸部疾病分类算法RMS-Net.首先,该算法将优化后的残差收缩模块加入到基础网络MobileViT中,利用软阈值化的方式过滤X光片中的背景噪声;其次,提出混合知识蒸馏训练策略,利用多层级注意力图和相似性激活矩阵作为监督信号,提升轻量级模型的分类性能;最后,使用焦点损失函数(focal loss,FL)缓解数据集正负样本不均衡的问题.在ChestX-ray14数据集上展开验证,蒸馏训练后的RMSNet学生模型识别14类胸部疾病的平均AUC值为0.836,而参数量和浮点计算量分别为0.96 M和0.27 G.实验结果表明,本文算法在保持轻量化的同时分类精度更高,能有效降低算法运行时的硬件要求.
Abstract
Existing lightweight networks for classifying thoracic diseases have a large number of parameters and require significant hardware resources.This paper proposes a lightweight algorithm for classifying thoracic diseases based on mixed knowledge distillation(KD)training strategy.Firstly,the algorithm incorporates an optimized residual shrinkage module into the MobileViT base network and employs soft thresholding to filter background noise in X-ray images.Then a mixed knowledge distillation training strategy is proposed,utilizing multi-level attention maps and similarity activation matrices as supervisory signals to enhance the ability of lightweight networks to recognize thoracic diseases.Finally,the focal loss function is employed to address the imbalance between positive and negative samples in the dataset.Experimental results on the ChestX-Ray14 dataset demonstrate that the average AUC value for the RMSNet student model trained with distilled knowledge to recognize 14 types of thoracic diseases is 0.836.The number of parameters and FLOPs are only 0.96 M and 0.27 G,respectively.These results indicate that the proposed algorithm improves classification accuracy while maintaining lightweight,enabling the network to run with less hardware.
关键词
ChestX-ray14/多标签分类/卷积神经网络(CNN)/医学图像处理/知识蒸馏(KD)Key words
ChestX-ray14/multi-label classification/convolutional neural network(CNN)/medical image processing/knowledge distillation(KD)引用本文复制引用
基金项目
国家自然科学基金(61471263)
国家自然科学基金(62272337)
天津市自然科学基金(16JCZDJC31100)
天津大学自主创新基金(2021XZC-0024)
出版年
2024
NETL
NSTL
万方数据