Multi-source Remote Sensing Image Fusion Based on Triple-branch Generative Adversarial Network Constructed by CNN and ViT
Multi-source remote sensing images have complementary information,where panchromatic images of the same scene have higher spatial resolution,while multispectral images have higher spectral resolution.Through multimodal image fusion technology,the information of panchromatic images and multispectral images can be integrated to obtain a fused image with both high spatial resolution and high spectral resolution.To this end,this paper proposes a multi-source remote sensing image fusion method using a triple-branch generative adversarial network constructed by CNN and ViT.Specifically,the pan-chromatic image and multispectral image are firstly input into the spatial and spectral branches of the generator for feature extraction,respectively.At the same time,the panchromatic image and multispec-tral image are connected and input into the fusion branch of the generator for feature extraction;Then,in the fusion branch,the features extracted from the spatial and spectral branches are gradually interacted and connected with the features extracted from the fusion branch,and the fused image is reconstructed.Next,two discriminators,namely spatial discriminator and spectral discriminator,are used to distin-guish the authenticity of the fused image from both spatial and spectral information aspects.Finally,through adversarial training between the generator and two discriminators,a fused image with both high spatial resolution and high spectral resolution is obtained.Experimental results show that compared with CNMF,PanNet,Pan-GAN,SDPNet methods,the fusion results obtained by the proposed method are superior in both qualitative evaluations and quantitative assessments.
NETL
NSTL
万方数据
