首页|基于双分支交互的实时语义分割算法

基于双分支交互的实时语义分割算法

扫码查看
原文链接
  • 国家科技期刊平台
  • NETL
  • NSTL
  • 万方数据
  • 维普
针对双分支实时语义分割算法存在双分支交互差、多尺度上下文信息提取不完善等问题,提出了基于双分支交互的实时语义分割网络(dual-branch interactive multi-scale fusion network for real-time semantic segmentation,DIMFNet).算法以引导聚合双边语义分割网络(bilateral network with guided aggregation for real-time semantic segmentation,BiseNetV2)的双分支结构为基准进行改进,空间分支提取空间细节特征,上下文分支提取深层上下文特征.结合注意力思想提出注意力引导高级语义融合模块(attention guide high-level semantics fusion module,AGHSM)实现双分支的交互融合,以获得更好的空间特征表示;对金字塔池化模块进行改进,提出采用多层聚合金字塔池化模块(multi-layer aggregation pyramid pooling module,MAPPM)提取多尺度上下文特征,以获得更好的上下文特征表示.算法在Cityscapes数据集上进行消融实验并与现有实时语义分割网络进行对比,验证了各模块的有效性,以 124.5 f/s达到了 77.9%的平均交并比(mean intersection over union,MIoU);在CamVid数据集上以 211.1 f/s达到了 75.1%的MIoU.相比现有的实时语义分割网络,本文算法更好地权衡了分割的精度和速度.
Real-time semantic segmentation algorithm based on two-branch interaction
In response to current issues of the dual-branch real-time semantic segmentation algorithm,such as poor interaction between the two branches and incomplete extraction of multi-scale contextual information,this paper proposes the dual-branch interactive multi-scale fusion network for real-time semantic segmentation(DIMFNet).The algorithm is based on the dual-branch structure of the bilateral network with guided aggregation for real-time semantic segmentation(BiseNetV2),with the spatial branch extracting spatial detail features and the context branch extracting deep contextual features.An attention guided high-level semantics fusion module(AGHSM)is proposed to achieve interactive fusion of the dual branches using the attention mechanism,so as to obtain better spatial feature representation.Furthermore,the algorithm improves the pyramid pooling module and introduces the multi-layer aggregation pyramid pooling module(MAPPM)to extract multi-scale contextual features,obtaining better contextual feature representation.The algorithm conducts ablation experiments on the Cityscapes dataset and is compared with existing real-time semantic segmentation networks,verifying effectiveness of each module.It achieves an average intersection over union(MIoU)of 77.9%at a speed of 124.5 frames per second(f/s)on the Cityscapes dataset,and 75.1%MIoU at 211.1 f/s on the CamVid dataset.Compared with existing real-time semantic segmentation networks,the proposed algorithm can better balance segmentation accuracy and speed.

real-time semantic segmentationspatial branchcontext branchfeature fusionattention mechanismmulti-scale feature extractionpooling pyramids moduledeep supervision

杨迪、陈春雨

展开 >

哈尔滨工程大学信息与通信工程学院,黑龙江哈尔滨 150001

实时语义分割 空间分支 上下文分支 特征融合 注意力机制 多尺度特征提取 池化金字塔 深度监督

国家自然科学基金中央高校基本科研业务费专项

618711423072020CFT0803

2024

10.11991/yykj.202301006

应用科技
哈尔滨工程大学

应用科技

CSTPCD
影响因子：0.693
ISSN：1009-671X
年,卷(期)：2024.51(2)
  • 24