分级监督范式指导下的遥感图像超分辨率方法
Remote sensing image super-resolution guided by multi-level supervision paradigm
李明锴 1徐其志1
作者信息
- 1. 北京理工大学 机电学院,北京 100081
- 折叠
摘要
超分辨率技术可提升遥感图像空间分辨率,为基于遥感图像的目标检测、场景分类等任务提供更加清晰的数据集,具有广泛的应用价值.然而,现有基于深度学习的超分辨率方法存在监督次数不足的问题,导致超分辨率重建图像中易出现细节损失和伪细节.针对这一问题,本文提出基于分级监督的遥感图像超分辨率方法(MSSR).首先,提出了一个分级监督网络架构,通过引入多级真值图像作为监督,为超分辨率过程提供充足的图像细节恢复指引,进而减少超分辨率结果中细节损失和伪细节的出现.其次,为了便于构建级数可变、超分辨率倍数可变的分级监督网络,设计了一个轻量化的、超分辨率倍数可灵活调整的同构超分辨率模块(BSRC).各级BSRC的网络结构基本相同,便于迁移网络权重,缩短训练时间.最后,针对分级网络超分辨率倍数一定时,网络级数及各级分辨率倍数有多种组合方式的问题,对比多种分级方式下的超分辨率结果,给出最佳网络分级方式.此外,构建了一个包含世界各地复杂细节地面场景的遥感图像数据集(RSSRD).在该数据集和UCMerced、AID两个公开数据集上进行超分辨率实验,实验结果显示本文方法优于现有常用超分辨率方法.
Abstract
Super-resolution improves the spatial resolution of remote sensing images,providing detailed information for various satellite applications.However,existing methods often generate pseudo-detail and lose true detail in reconstructed images due to insufficient supervision images.To address this issue,a progressive super-resolution method based on multilevel supervision structure(MSSR)was proposed.First,the MSSR introduced ground truth images as guides,which reduced the loss of true detail and mitigated the appearance of pseudo-detail in an output image.The MSSR network consists of several basic super-resolution components(BSRCs)and multilevel supervision.The overall super-resolution scale factor of the MSSR network can be set flexibly.The BSRCs can be increased or decreased,similar to building blocks,and BSRCs decrease with decreasing overall super-resolution scale factor and increase with increasing overall super-resolution scale factor.The scale factor of each BSRC is determined by the overall scale factor and the number of BSRCs.Second,a scale-factor-adjustable and lightweight basic super-resolution component was designed to enable the construction of multilevel supervision networks with different number of BSRCs and different scale factors,such as building blocks.The BSRC consists of a multiscale feature extraction module,a global feature extraction module and an image reconstruction module.Given that the scale factor of each BSRC should have a degree of flexibility,the network structure of each BSRC is the same except for the image reconstruction module.This approach shortens the overall training time of the network.Finally,a method of dividing super-resolution overall scale factor was proposed,and the effects of different number and different scale factors of BSRCs on the performance of multilevel supervision network were explored.For the super-resolution process with a certain scale factor,we need a method to divide the overall scale factor into each BSRC.The number of supervision increases with the number of BSRCs,and the super-resolution ill-posedness is reduced.In addition,the total number of network layers and the number of computations increase.We determined the optimal number of BSRCs and their respective super-resolution scale factors by comparing the super-resolution effects of multiple BSRC combinations.Additionally,a new remote sensing dataset containing worldwide scenes was constructed for the super-resolution task in this paper.To adequately train and test the proposed super-resolution method and existing methods,we used our datasets and two existing super-resolution datasets:the UCMerced and AID datasets.We compared our method with the state-of-the-art methods:VDSR,SRGAN,RDN,RCAN,DRN,and TransENet.The experiment results on three datasets demonstrated that our MSSR network outperformed these methods.The progressive network and multilevel supervision structure can effectively suppress super-resolution ill-posedness and reduce pseudo-detail and detail loss in super-resolution results.By further analyzing the experimental results,we found that multilevel supervision has greater performance gains in super-resolution tasks with a larger scale factor than other methods.We speculated that the multilevel supervision network exhibit improved performance in super-resolution tasks at 4× magnification.In future research,we will explore multilevel supervision networks in super-resolution tasks at large magnifications(i.e.,5× and 8×).
关键词
遥感图像/深度学习/分级监督/超分辨率/多尺度特征提取/渐进式网络/参数共享/迁移学习Key words
remote sensing image/deep learning/multi-level supervision/super-resolution/multi-scale feature extration/progressive network/parameter sharing/transfer learning引用本文复制引用
基金项目
国家自然科学基金(61972021)
国家自然科学基金(61672076)
出版年
2024
NETL
NSTL
万方数据