基于多模态多示例学习的免疫介导性肾小球疾病自动分类方法
Automatic classification of immune-mediated glomerular diseases based on multi-modal multi-instance learning
龙楷兴 1翁丹仪 1耿舰 2路艳蒙 3周志涛 3曹蕾1
作者信息
- 1. 南方医科大学,生物医学工程学院//广东省医学图像处理重点实验室//广东省医学成像与诊断技术工程实验室,广东 广州 510515
- 2. 南方医科大学,基础医学院,广东 广州 510515;广州华银医学检验中心,广东 广州 510515
- 3. 南方医科大学,中心实验室,广东 广州 510515
- 折叠
摘要
目的 探讨如何利用多模态深度学习方法,联合光学显微镜(OM)、免疫荧光显微镜(IM)及透射电子显微镜(TEM)对应的3种图像进行免疫介导性肾小球疾病分类.方法 基于273例患者的病理图像进行回顾性研究,构建多模态多示例模型对3种免疫介导性的肾小球疾病——免疫球蛋白A肾病(IgAN)、膜性肾病(MN)、狼疮性肾炎(LN)进行分类.该模型采用示例水平的多示例学习(I-MIL)方法挑选患者的TEM图像并与同一患者的OM图像和IM图像进行多模态特征融合.通过该模型与单模态、双模态模型的比较,探究3种模态之间的不同组合形式以及模态特征融合方式的特性.结果 联合OM、IM以及TEM图像建立的多模态多示例模型准确率为(88.34±2.12)%,优于准确率为(87.08±4.25)%的最优的单模态模型,以及准确率为(87.92±3.06)%的最优的双模态模型.结论 本研究成功建立基于OM、IM及TEM三种模态图像的多模态多示例模型,并验证了采用多示例学习结合多模态学习方法对免疫介导性肾小球疾病分类的有效性.
Abstract
Objective To develop a multi-modal deep learning method for automatic classification of immune-mediated glomerular diseases based on images of optical microscopy(OM),immunofluorescence microscopy(IM),and transmission electron microscopy(TEM).Methods We retrospectively collected the pathological images from 273 patients and constructed a multi-modal multi-instance model for classification of 3 immune-mediated glomerular diseases,namely immunoglobulin A nephropathy(IgAN),membranous nephropathy(MN),and lupus nephritis(LN).This model adopts an instance-level multi-instance learning(I-MIL)method to select the TEM images for multi-modal feature fusion with the OM images and IM images of the same patient.By comparing this model with unimodal and bimodal models,we explored different combinations of the 3 modalities and the optimal methods for modal feature fusion.Results The multi-modal multi-instance model combining OM,IM,and TEM images had a disease classification accuracy of(88.34±2.12)%,superior to that of the optimal unimodal model[(87.08±4.25)%]and that of the optimal bimodal model[(87.92±3.06)%].Conclusion This multi-modal multi-instance model based on OM,IM,and TEM images can achieve automatic classification of immune-mediated glomerular diseases with a good classification accuracy.
关键词
肾活检病理/肾小球疾病/深度学习/多模态融合/多示例学习Key words
renal biopsy pathology/glomerular disease/deep learning/multi-modal fusion/multi-instance learning引用本文复制引用
出版年
2024
国家科技期刊平台
NSTL
万方数据