Towards imbalanced motion:part-decoupling network for video portrait segmentation

扫码查看

原文链接

NETL
NSTL
万方数据

外文摘要：Video portrait segmentation(VPS),aiming at segmenting prominent foreground portraits from video frames,has received much attention in recent years.However,the simplicity of existing VPS datasets leads to a limitation on extensive research of the task.In this work,we propose a new intricate large-scale multi-scene video portrait segmentation dataset MVPS consisting of 101 video clips in 7 scenario categories,in which 10843 sampled frames are finely annotated at the pixel level.The dataset has diverse scenes and complicated background environments,which is the most complex dataset in VPS to our best knowledge.Through the observation of a large number of videos with portraits during dataset construction,we find that due to the joint structure of the human body,the motion of portraits is part-associated,which leads to the different parts being relatively independent in motion.That is,the motion of different parts of the portraits is imbalanced.Towards this imbalance,an intuitive and reasonable idea is that different motion states in portraits can be better exploited by decoupling the portraits into parts.To achieve this,we propose a part-decoupling network(PDNet)for VPS.Specifically,an inter-frame part-discriminated attention(IPDA)module is proposed which unsupervisedly segments portrait into parts and utilizes different attentiveness on discriminative features specified to each different part.In this way,appropriate attention can be imposed on portrait parts with imbalanced motion to extract part-discriminated correlations,so that the portraits can be segmented more accurately.Experimental results demonstrate that our method achieves leading performance with the comparison to state-of-the-art methods.

外文关键词：

video portrait segmentationimbalanced motionunsupervised part decouplingmotion corre-lationinter-frame attention

作者：

Tianshu YU、Changqun XIA、Jia LI

展开 >

作者单位：

State Key Laboratory of Virtual Reality Technology and Systems,School of Computer Science and Engineering,Beihang University,Beijing 100191,China

Peng Cheng Laboratory,Shenzhen 518055,China

基金：

National Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaMajor Key Project of PCL

项目编号：

6213200262102206PCL2023A10-1

出版年：

2024

DOI：

10.1007/s11432-023-4030-y

中国科学:信息科学(英文版)

中国科学院

中国科学:信息科学(英文版)

CSTPCDEI

影响因子：0.715

ISSN：1674-733X

年,卷(期)：2024.67(7)