Research on Multimodal Knowledge Graph Construction Method for Chinese Electronic Medical Record
[Purpose/Significance]To effectively sequence and organize the multimodal electronic medical re-cord data with complex structure and strong specialization,in this paper,a multimodal knowledge graph is constructed for Chinese electronic medical record based on multimodal deep learning and natural language processing methods.It provides a more comprehensive support for the clinical decision support and domain knowledge service.[Method/Process]Firstly,the multimodal data were preprocessed to construct a multimodal electronic medical record dataset.Then the medical image and text features were extracted respectively using ResNet and BERT pre-training models.The entity-relationship joint extraction model was built through a multimodal fusion method based on a multilevel vi-sual prompt mechanism.Finally,a graph database was used to store and visually presented the knowledge contained in the different modalities of the healthcare field.[Result/Conclusion]This research constructs a multimodal knowledge graph for Chinese electronic medical records.The graph further enriches the domain knowledge association of text and image modalities,advances the practical scope of multimodal knowledge organization.It provides a foundation for knowledge sequencing,precise knowledge service,and deep knowledge discovery in the healthcare domain.
multimodal knowledge graphmultimodal information extractionelectronic medical recordknowledge organization
万方数据
维普
