基于动态采样对偶可变形网络的实时视频实例分割

扫码查看

原文链接

万方数据
维普

中文摘要：为了更好地利用视频帧中蕴含的时间信息，提升视频实例分割的推理速度，提出动态采样对偶可变形网络(DSDDN)。DSDDN使用动态采样策略，根据前、后帧的相似性调整采样策略。对于相似性高的帧，该方法跳过当前帧的推理过程，仅使用前帧分割进行简单迁移计算。对于相似性低的帧，该方法动态聚合时间跨度更大的视频帧作为输入，对当前帧进行信息增强。在Transformer结构里，该方法额外使用2个可变形操作，避免基于注意力的方法中的指数级计算量。提供精心设计的追踪头和损失函数，优化复杂的网络。在YouTube-VIS数据集上获得了39。1％的平均推理精度与40。2帧/s的推理速度，验证了提出的方法能够在实时视频分割任务上取得精度与推理速度的良好平衡。

外文标题：Dynamic sampling dual deformable network for online video instance segmentation

外文摘要：The dynamic sampling dual deformable network(DSDDN)was proposed in order to enhance the inference speed of video instance segmentation by better using temporal information within video frames.A dynamic sampling strategy was employed,which adjusted the sampling policy based on the similarity between consecutive frames.The inference process for the current frame was skipped for frames with high similarity by utilizing only segmentation results from the preceding frame for straightforward transfer computation.Frames with a larger temporal span were dynamically aggregated for frames with low similarity in order to enhance information for the current frame.Two deformable operations were additionally incorporated within the Transformer structure to circumvent the exponential computational cost associated with attention-based methods.The complex network was optimized through carefully designed tracking heads and loss functions.The proposed method achieves an inference accuracy of 39.1％mAP and an inference speed of 40.2 frames per second on the YouTube-VIS dataset,validating the effectiveness of the approach in achieving a favorable balance between accuracy and speed in real-time video segmentation tasks.

外文关键词：

videoonline inferenceinstance segmentationdynamic networkdual deformable network

作者：

宋一然、周千寓、邵志文、易冉、马利庄

展开 >

作者单位：

上海交通大学计算机科学与工程系,上海 200240

中国矿业大学计算机科学与技术学院,江苏徐州 221116

关键词：

视频实时推理实例分割动态网络对偶可变形网络

基金：

上海市科委项目国家自然科学基金Shanghai Sailing ProgramCCF-Tencent Open Research FundYoung Elite Scientists Sponsorship Program by CAST国家自然科学基金

项目编号：

215111012007219282122YF1420300RAGR202201212022QNRC00162302297

出版年：

2024

DOI：

10.3785/j.issn.1008-973X.2024.02.003

浙江大学学报(工学版)

浙江大学

浙江大学学报(工学版)

CSTPCD北大核心

影响因子：0.625

ISSN：1008-973X

年,卷(期)：2024.58(2)

参考文献量35