Multimodal medical image fusion method based on structural functional cross neural network
To solve the problems of texture detail blurring and low contrast in multimodal medical image fusion,a multimodal medical image fusion method with structural-functional crossed neural networks was proposed.Firstly,this method designed a structural and functional cross neural network model based on the structural and functional information of medical images.Within each structural-functional cross mod-ule,a residual network model was also incorporated.This approach not only effectively extracted the structural and functional information from anatomical and physiological medical images but also facilitated interaction between structural and functional information.As a result,it effectively captured texture details from multi-source medical images,creating fused images that closely align with human visual characteris-tics.Secondly,a new attention mechanism module was constructed by utilizing the effective channel atten-tion mechanism and spatial attention mechanism model(ECA-S),which continuously adjusted the weights of structural and functional information to fuse images,thereby improving the contrast and contour information of the fused image,and to make the fused image color more natural and realistic.Finally,a decomposition process from the fused image to the source image was designed,and since the quality of the decomposed image depends directly on the fusion result,the decomposition process could make the fused image contain more texture detail information and contour information of the source image.By comparing with seven high-level methods for medical image fusion proposed in recent years,the objective evaluation indexes of AG,EN,SF,MI,QAB/F and CC of this paper's method are improved by 22.87%,19.64%,23.02%,12.70%,6.79%and 30.35%on average,respectively,indicating that this paper's method can obtain fusion results with clearer texture details,higher contrast and better contours in subjective visual and objective indexes are better than other seven high-level contrast methods.
multimodal medical image fusionstructural and functional information cross-interacting net-workattention mechanismdecomposition network
万方数据
维普
