Two-discriminators-deep residual GAN hyperspectral image pan-sharpening
Objective Hyperspectral image(HS)pan-sharpening can obtain the fused image with both high spatial resolu-tion and high spectral resolution by using the complementary information between the high spectral resolution HS and the high spatial resolution multi-spectral image(MS).However,HS and MS images both have multi-bands,which is different from the traditional MS pan-sharpening where one band panchromatic image(PAN)with multi-band MS image.This many-to-many band relationship is not convenient for the implementation of pan-sharpening method.Moreover,for those bands where HS exceeds the spectral range of MS,there will be obvious spectral distortion in the fused image due to the lack of strict physical complementary information.In order to solve the above problems,this paper exploits the data-driven advan-tages of deep learning and proposes the two discriminators deep residual generative adversarial network(2DDRGAN)for hyperspectral image pan-sharpening based on Gaofen-5(GF-5)HS image and Sentinel-2 MS image.Method Considering the spectral range relationship between HS and MS,this paper adopts the grouping pan-sharpening strategy to transform the many-to-many problem into multiple one-to-many problems by using the correlation between bands of HS and MS.For the HS band within the spectral range of the high spatial resolution MS image,the band of HS image is directly assigned to the group corresponding to the high spatial resolution MS image.For the HS bands outside the spectral range,the correlation coefficient is used as the band grouping standard to group the bands of HS and MS.It solves the problem that two multi-band images are not easy to be fused directly,and indirectly improves the spectral fidelity of the fused image.The proposed 2DDRGAN consists of one generation network and two discrimination networks.The generation network mainly extracts the deep spectral and spatial features by establishing the deep residual module.Two discrimination networks judge the spatial quality and the spectral quality of the fused image,respectively,to improve the quality of the output fused image from the generation network.The main task of the spatial discrimination network is to compare the results of the generated network with the high spatial resolution MS image,so as to ensure that the generated fused images have high spatial resolution.The main task of spectral discrimination network is to compare the results of the generated network with the HS images,to ensure the generated fused images have high spectral resolution.Moreover,the deep learning pan-sharpening methods do not have real high spatial resolution and high spectral resolution images as the fusion result labels.Most of them currently are based on the simulated data made by Wald protocol.In this paper,the deep learning network does not need to create additional fusion result labels.The images to be fused themselves are labels,which greatly reduces the workload of hyperspectral fusion with a huge amount of data,and is also a fundamental change in the current deep learning fusion.Result The fusion results of the proposed 2DDRGAN method in different scenarios are compared with traditional methods and existing deep learning methods.The experimental results show that the 2DDRGAN method has high spectral fidelity while improving the spatial resolution.The spectral curve evaluation also verifies shows that the 2DDRGAN network has good spectral fidelity for hyperspectral image bands beyond the spectral range of high spatial resolution images.Conclusion This method extracts the spectral features of hyperspectral images and the spatial features of high spatial resolution images through the depth residual module,and introduces the double discrimination network,so that the fusion results can better improve the spatial information while maintaining the spectral information.
NETL
NSTL
万方数据
