基于深度学习多模态融合的2型糖尿病中医证素辨证模型的构建
Construction of a Chinese Medicine Zhengsu Differentiation Model for Type 2 Diabetes Based on Deep Learning Multimodal Fusion
赵智慧 1周毅 2李炜弘 3汤朝晖 3郭强 4陈日高2
作者信息
- 1. 成都中医药大学智能医学学院 成都 610075
- 2. 成都中医药大学附属医院 成都 610072
- 3. 成都中医药大学基础医学院 成都 610075
- 4. 成都市中西医结合医院 成都 610095
- 折叠
摘要
目的 为适应互联网+智能医疗的时代需求,纳入舌诊仪图像数据及问诊结构化数据,采用深度学习、多模态融合等方法构建2型糖尿病中医证素辨证模型,为中医智能化辨证提供实验支撑和科学依据.方法 共纳入2585例2型糖尿病患者,邀请3位专家分别进行证素辨证标记.基于深度全连接神经网络、U2-Net与ResNet34等网络构建基于舌图数据、症候数据的症候辨证模型(S-Model)、舌图辨证模型(T-Model),并采用多模态融合技术构建以二者为共同输入的多模态融合辨证模型(TS-Model).通过F1值、精确率、召回率等对比不同模型预测性能.结果 T-Model对十四类证素的预测F1值波动于0.000%-86.726%,S-Model的预测F1值波动于0.000%-97.826%,TS-Mode的预测F1值波动于55.556%-99.065%.与T-Model、S-Model对比,TS-Model整体F1值较高且稳定.结论 基于深度学习多模态融合技术构建中医证素智能辨证模型性能较好.多模态融合技术适用于中医证素辨证模型优化,为下一步建立四诊信息全客观化的高度智能证素辨证模型提供方法学支持.
Abstract
Objective To construct a TCM Zhengsu differentiation model for type 2 diabetes based on deep learning and multimodal fusion,thus providing algorithmic support for full intelligence in TCM Zhengsu differentiation.Methods A total of 2585 patients with type 2 diabetes were recruited.Three experts were invited to perform the Zhengsu differentiation separately.Deep fully connected neural networks,U2-Net and ResNet34 networks were applied to construct the symptom-based differentiation model(S-Model)and the tongue image-based differentiation model(T-Model),respectively,while multimodal fusion techniques were employed to build the multimodal fusion model(TS-Model)with the above two as co-inputs.Finally,the prediction performance of the above models was compared by F1 value,accuracy,and recall.Results The predicted F1 values of the T-Model fluctuated from 0.000%to 86.726%,while those in the S-Model and TS-Model fluctuated from 0.000%to 97.826%and from 55.556%to 99.065%,respectively.A stable and high F1 value was found in the TS-Model.Conclusion The multimodal fusion technique was demonstrated to be applicable in the TCM Zhengsu differentiation model,which provided methodological support for developingof a fully intelligent Zhengsu differentiation model with high objective four diagnostic information.
关键词
证素辨证/2型糖尿病/深度学习/多模态融合Key words
Zhengsu differentiation/Type 2 diabetes/Deep learning/Multimodal fusion引用本文复制引用
基金项目
国家科学技术部重点研发计划(2017YFC1703304)
中国博士后科学基金(2022MD723720)
成都中医药大学校基金博士后专项(BSH2023026)
出版年
2024
NETL
NSTL
万方数据