遥感信息2024,Vol.39Issue(1) :10-17.DOI:10.20091/j.cnki.1000-3177.2024.01.002

基于双流多模态多层融合网络的地基云分类方法

Ground-based Cloud Classification Method Based on Dual-flow Multi-modal Multi-layer Fusion Network

王敏 李晟 庄志豪 周树道 王康
遥感信息2024,Vol.39Issue(1) :10-17.DOI:10.20091/j.cnki.1000-3177.2024.01.002

基于双流多模态多层融合网络的地基云分类方法

Ground-based Cloud Classification Method Based on Dual-flow Multi-modal Multi-layer Fusion Network

王敏 1李晟 2庄志豪 2周树道 3王康2
扫码查看

作者信息

  • 1. 南京信息工程大学电子与信息工程学院,南京 210044;安徽建筑大学电子与信息工程学院,合肥 230009
  • 2. 南京信息工程大学电子与信息工程学院,南京 210044
  • 3. 国防科技大学气象与海洋学院,长沙 410003
  • 折叠

摘要

地基云的准确分类对于天气预报、航空航天等多个领域具有重要意义.近年来,深度学习在地基云分类领域取得了卓越的成果,但除地基云的视觉特征外,地基云的辅助特征,即地基云多模态信息,对于地基云分类也有重要作用.针对地基云多模态特征信息的挖掘和融合研究,提出了 一种基于双流多模态多层融合网络(dual-flow multi-modal multi-layer fusion network,DMMFN)的地基云分类方法,首次将多模态信息分开传递进不同子网络,并在特征层进行异构特征融合,最终该模型在多模态地基云数据集上得到85.70%的高准确率.实验结果表明,所提出的DMMFN网络模型能够有效地将地基云多模态信息与视觉特征结合,提升地基云分类的准确率.

Abstract

The accurate classification of ground-based cloud is of great significance to many fields such as weather forecasting,aerospace and so on.In recent years,deep learning has achieved remarkable achievements in the classification of ground-based cloud.However,in addition to the visual features of ground-based cloud,the auxiliary features of ground-based cloud,namely the multi-modal information of ground-based cloud,also play an important role in the classification of ground-based cloud.In order to mine and integrate the multi-modal feature information of ground-based cloud,this study designs a ground-based cloud classification method based on dual-flow multi-modal multi-layer fusion network(DMMFN).Firstly,multi-modal information is separately transmitted into different sub-networks.Secondly,heterogeneous feature fusion is carried out in the feature layer.Finally,the model achieves a high accuracy rate of 85.70%on the multimodal ground-based cloud dataset.The experimental results show that the proposed DMMFN network model can effectively combine ground-based cloud multi-modal information with visual features,and improve the accuracy of ground-based cloud classification.

关键词

地基云分类/卷积神经网络/注意力机制/多模态信息/特征融合

Key words

ground-based cloud classification/convolutional neural network/attention mechanism/multi-modal information/feature fusion

引用本文复制引用

基金项目

国家自然科学基金(41775165)

国家自然科学基金(41775039)

安徽省高等学校杰出青年科研项目(2023AH020022)

江苏省研究生科研与实践创新计划(KYCX23_1364)

出版年

2024
遥感信息
科学技术部国家遥感中心,中国测绘科学研究院

遥感信息

CSTPCDCSCD北大核心
影响因子:0.712
ISSN:1000-3177
参考文献量22
段落导航相关论文