首页|自注意力映射超时相遥感图像超分辨率重建

自注意力映射超时相遥感图像超分辨率重建

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
针对视频卫星超时相数据高时间分辨率的特点,单帧超分辨率重建算法无法充分利用其时空信息,提出一种基于自注意力的超时相遥感图像超分辨率重建模型.对超时相序列帧先依据帧速率划分为多个时间组,利用3D卷积对空间及时间同时建模的特点提取不同时间组下的时空信息,之后通过改进的广泛自注意力残差块增加注意力计算范围并完成高动态映射,在完成超时相序列帧建模的同时提取其丰富的空间细节信息,最后将多个时间组的特征融合,通过亚像素卷积提升分辨率并完成重建.所提算法的优点在于,对超时相数据的多时间组处理充分利用了其丰富的时空信息,改进的自注意力块可在无配准情况下完成序列帧的建模并提高空间细节信息的提取能力.在高分四号数据上的实验表明,本文算法的主观视觉效果及客观评价指标均优于对比算法,在高分四号数据两倍上采样重建时,较双三次插值算法在PSNR值上提升了 2.49 dB以上,且在4倍重建时依然有很强的重建性能,较双三次插值算法有巨大提升.实验结果表明,该方法拥有较好的超分辨率重建效果,有利于超时相数据在各领域的应用.
Super-resolution reconstruction of hypertemporal remote sensing images based on self-attention
Video satellite hypertemporal data have the characteristics of high temporal resolution,while the single-frame super-resolution reconstruction algorithm can only use the information of the image frame itself,and the reconstruction effect is limited.Therefore,how to utilize fully and effectively the rich spatiotemporal information in hypertemporal data in the super-resolution reconstruction of video satellite images is an issue of interest.Aiming at the characteristics of hypertemporal data,this study proposes a self-attention-based super-resolution reconstruction model of hypertemporal remote sensing images.The model can mine high-frequency information from low-resolution images through an end-to-end network.High-resolution images are recovered from multiple frames of low-resolution images.First,the hypertemporal sequence frames are divided into multiple time groups according to the frame rate,and the spatial and temporal information under different time groups are extracted by using the characteristics of 3D convolution to model space and time simultaneously.It pays attention to the calculation range and completes high dynamic mapping,extracts rich spatial detail information while realizing hypertemporal sequence frame modeling,and finally fuses the features of multiple time groups and completes the reconstruction through subpixel convolution to improve the resolution.The advantage of the proposed algorithm is that the multitime group feature fusion method can extract the spatiotemporal information of sequence frames in multiple time dimensions and fully mine the rich spatiotemporal-related information in the hypertemporal data;the improved self-attention block can be used without registration.It can complete the modeling of sequence frames and improve the extraction ability of detailed spatial information.Experiments on the GF-4 dataset show that the subjective visual effect and objective evaluation index of the algorithm are better than those of the comparison algorithm.When the GF-4 dataset is reconstructed twice,the PSNR value is improved by more than 2.49 dB compared with the bicubic interpolation algorithm,and it still has a strong reconstruction performance when the reconstruction is four times,which is a huge improvement compared with the bicubic interpolation algorithm.Experimental results show that the method has good super-resolution reconstruction effect,which is beneficial to the application of hypertemporal data in various fields.The self-attention-based super-resolution model of hypertemporal remote sensing images proposed in this study fully extracts the spatiotemporal information in hypertemporal data by dividing multiple time groups and calculating the attention features in each time group.The combination of multitemporal group feature fusion and self-attention enables the modeling of overphase sequence frames while ensuring the ability to extract detailed information.Comparative experiments on the GF-4 dataset show that the algorithm in this study is superior to the compared algorithm in terms of objective evaluation indicators and subjective visual effects,verifying the effectiveness and advancement of the algorithm in super-resolution reconstruction of hypertemporal data and the algorithm's improved reconstruction performance.However,the proposed algorithm must be optimized in terms of calculation time.In a follow-up research,the algorithm structure will be optimized(e.g.,changing the residual structure of the network)to reduce the calculation time while ensuring accuracy.

remote sensinghyper-temporal datasuper-resolution reconstructiondeep learningfusion feature of multiple time groupswide self-attentionGF-4

唐晓天、杨雪、李峰、马骏、梁亮

展开 >

中国空间技术研究院钱学森空间技术实验室,北京 100094

河南大学软件学院,开封 475004

南京大学国际地球系统科学研究所,南京 210023

清华大学电子工程系,北京 100084

展开 >

遥感 超时相数据 超分辨率重建 深度学习 多时间组融合特征 广泛自注意力 高分四号

国家重点研发计划高分辨率对地观测系统重大专项

2020YFA071410030-Y30F06-9003-20/22

2024

10.11834/jrs.20221825

遥感学报
中国地理学会环境遥感分会 中国科学院遥感应用研究所

遥感学报

CSTPCD北大核心
影响因子：2.921
ISSN：1007-4619
年,卷(期)：2024.28(6)