Adaptive Fusion Model for Near-infrared and Visible Light Images Based on Multimodal Sensors
Aiming at the shortcomings of feature extraction and fusion strategies in the existing image fusion methods,this paper proposes an adaptive fusion model for near-infrared and visible light images,called STAFuse,based on frequency domain decomposition.It realizes the effective fusion of different modal image features,by introducing feature extraction modules of Transformer and CNN and the adaptive fusion modules.To address the issues of large size and complex calibration in traditional multi-sensor systems on the acquisition of the multimodal images,a novel multimodal sensor is designed,capable of simultaneously capturing high-resolution visible light images and low-resolution near-infrared images.Experimental results demonstrate that STAFuse outperforms existing models in multiple metrics,which improves by 102.7%compared with DenseFuse model in Structural Similarity(SSIM),improves by 25%compared with DIDFuse model in Visual Information Fidelity(VIF),and is outstanding in maintaining visual quality and image details.
near-infrared and visible light fusionadaptive fusionTransformerCNNmultimodal sensorfrequency domain decomposition
国家科技期刊平台
NSTL
万方数据
