Multimodal Medical Image Fusion Based on GAN and Multiscale Spatial Attention
Aiming to address the problem of multi-scale feature and texture detail information loss in the process of multimodal medical image fusion,a novel image fusion algorithm based on generative adversarial network (GAN) and multi-scale spatial attention mechanism was proposed. Firstly,the generator adopted an autoencoder structure to extract,fuse,and reconstructed the input images using an encoder and a decoder,generating the fused image. Secondly,the entire GAN framework employed a dual discriminator structure,enabling the generator to preserve salient features from multiple modal images in the fused image. Finally,a multi-scale spatial attention mechanism was constructed as a fundamental module for feature extraction in the encoder. It could effectively capture and re-tain multi-scale features from the source images,and incorporate spatial attention mechanism to better preserve the structures and details of the source images. Experimental results on the Whole Brain Atlas database demonstrated that the fused images generated by the proposed algorithm could exhibit richer texture details,enhancing human visual observation. Furthermore,the algorithm outperformed other advanced algorithms in such objective evaluation metrics as average gradient,peak signal-to-noise ratio,mutual information,and visual information fidelity for three different types of medical image fusion tasks,with average values of 0.3023,20.7207,1.4414,and 0.6498,respectively. Thus,the proposed algorithm demonstrated a certain advantage over other advanced algorithms.
image fusionmultimodal medical imagesgenerative adversarial networkfeature pyramidattention mechanism
万方数据
维普
