首页|面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究

面向肝细胞癌微血管侵犯评估的高效多模态贡献度感知网络研究

扫码查看
微血管侵犯(MicroVascular Invasion,MVI)是肝细胞癌(HepatoCellular Carcinoma,HCC)切除或移植患者出现早期复发和长期预后不良的重要影响因素,因此在HCC患者术前评估是否存在MVI具有非常重要的临床价值.近年来,深度学习为MVI影像诊断评估提供了有价值的解决方法,但受数据标注收集困难等因素的影响,目前研究多独立利用电子计算机断层扫描(Computed Tomography,CT)或核磁共振成像(Magnetic Resonance Imaging,MRI)手段采集影像中的单模态序列,缺乏对各成像手段中多模态序列的综合应用.在小样本场景下,为有效利用多模态序列数据,提高诊断效能,本文提出一种高效多模态贡献度感知网络.该网络可以利用模态分组卷积和高效多模态自适应加权模块,在极少计算开销的引入下,学习CT或MRI的各模态影像信息在复杂多样的MVI表象下的诊断贡献.本文在三甲医院收集的临床数据集上进行实验,结果表明该网络模型可以在少量有标注数据的支持下,取得优于多种基于注意力机制的神经网络模型的MVI诊断性能,为专业医师的诊断分析提供了有效参考.
Efficient Multimodal Contribution Aware Network for Assessment of Microvascular Invasion in Hepatocellular Carcinoma
Microvascular invasion (MVI) is an important factor for early recurrence and poor long-term prognosis in patients with hepatocellular carcinoma (HCC) after resection or transplantation. Therefore,it is of great clinical value to evaluate whether MVI exists in patients with HCC before operation. In recent years,deep learning has provided a valuable solution for MVI image diagnosis and evaluation. Nevertheless,due to the difficulties of data annotation and collection,the current researches mostly use computed tomography (CT) or magnetic resonance imaging (MRI) methods to collect single modal sequences in images independently,which lacks the comprehensive application of multimodal sequences in various imaging methods. In order to make more effective use of multimodal data of CT and MRI images and improve diagnosis ef-ficiency under few-shot scenarios,an efficient multimodal montribution aware network is proposed in this paper. The mo-dality grouping convolution and efficient multimodal adaptive weighting module in this network are used to to learn the di-agnostic contribution of each modal information of CT or MRI under complex and diverse MVI representation with little computational cost introduced. The experiment is carried out on the clinical dataset collected by the third-class hospital. Re-sult show that with the support of a small amount of labeled data,our method can achieve better MVI diagnostic perfor-mance than many deep neural networks based on attention mechanism,which provides an effective reference for profession-al doctors' diagnostic analysis.

microvascular invasion evaluationmultimodal fusionefficient multimodal contribution awaremodality grouping convolutionefficient multimodal adaptive weighting

贾熹滨、于高远、王珞、邓玉辉、杨大为、杨正汉

展开 >

北京工业大学信息学部,北京 100124

多媒体与智能软件技术北京市重点实验室(北京工业大学),北京 100124

北京人工智能研究院(北京工业大学),北京 100124

首都医科大学附属北京友谊医院放射科,北京 100050

展开 >

微血管侵犯评估 多模态融合 高效多模态贡献度感知 模态分组卷积 高效多模态自适应加权

国家自然科学基金国家自然科学基金国家自然科学基金北京市医管中心青苗人才项目

820718766217129861871276QML20200108

2024

电子学报
中国电子学会

电子学报

CSTPCD北大核心
影响因子:1.237
ISSN:0372-2112
年,卷(期):2024.52(6)