首页|集成CNN和Transformer的通道交互多层级融合变化检测

集成CNN和Transformer的通道交互多层级融合变化检测

Channel-interaction multi-level fusion network for change detection by integrating CNN and Transformer

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
为有效集成卷积神经网络的局部性和Transformer的全局性,提出一种全新的通道交互多层级融合变化检测网络CIMLFNet.以CNN和Transformer为基础,设计一种三通道特征提取器,以充分提取两期影像的时空特征;构建一种金字塔时空交叉注意力模块,利用通道2提取的特征增强通道1和3提取的特征,突出变化信息;提出一种双分支通道交互多层级融合模块,分别从层级优先和通道优先的角度对增强的特征进行融合,以充分利用CNN和Transformer的优势和互补性;给出一种简单有效的边界区域增强分类器.在WHU、Google、GVLM和LEVIR等4组公开变化检测数据上,CIMLFNet 的 F1/IoU 值分别达到 91.19％/83.80％、85.97％/75.40％、88.85％/79.94％和 90.07％/81.94％,明显优于6组对比方法,验证了 CIMLFNet的有效性.
This paper proposes a novel Channel-Interaction Multi-Level Fusion Network(CIMLFNet)for change detection(CD)based on parallel architecture,which integrates the local characteristics of convolutional neural network(CNN)and the global characteristics of Transformer effectively.First,based on complementary CNN and Transformer,a tripe-channel feature extractor is designed to fully extract the spatial-temporal features of bi-temporal images.Second,a Pyramid Spatial-Temporal Cross-Attention Module(PSTCAM)is constructed to highlight the change information.PSTCAM leverages the features extracted by Channel 2 to enhance the features extracted by Channels 1 and 3.Then,to make full use of the advantages and complementarity of CNN and Transformer,a dual-branch channel-interaction multi-level fusion module is proposed.It fuses the enhanced features from the perspectives of level-priority and channel-priority,respectively.Finally,a simple yet effective boundary-region-enhancement classifier is proposed.On the four pubic CD datasets,namely,WHU、Google、GVLM and LEVIR,the F1/IoU values of the proposed CIMLFNet are 91.19％/83.80％、85.97％/75.40％、88.85％/79.94％and 90.07％/81.94％,respectively,which are significantly better than the six comparison methods.The experimental results confirm the effectiveness of CIMLFNet.

remote sensing change detectionCNNTransformerchannel-interaction multi-level fusionpyramid spatial-temporal cross attentionboundary-region enhancement

邵攀、石卫超、秦道龙、张晓东、董婷、管宗胜

展开 >

三峡大学水电工程智能视觉监测湖北省重点实验室,湖北宜昌 443002

三峡大学计算机与信息学院,湖北宜昌 443002

武汉大学测绘遥感信息工程国家重点实验室,武汉 430079

遥感变化检测 CNN Transformer 通道交互多层级融合 金字塔时空交叉注意力 边界区域增强

国家自然科学基金项目湖北省自然科学基金项目

419013412024AFB867

2024

10.16251/j.cnki.1009-2307.2024.05.011

测绘科学
中国测绘科学研究院

测绘科学

CSTPCD北大核心
影响因子：0.774
ISSN：1009-2307
年,卷(期)：2024.49(5)