非约束环境下的轻量级实时多人三维动作捕捉
Lightweight multi-person motion capture system in the wild
杨文武 1李跃 1邢帅 1蔡佳航 1王勋1
作者信息
- 1. 浙江工商大学计算机科学与技术学院,杭州 310018
- 折叠
摘要
非接触式的多人三维动作捕捉技术在影视动画、游戏、虚实互动、体育运动和生物医学分析等领域有着广泛的应用.现有的国内外相关论文多数聚焦在算法创新,而缺少整体视角下的系统分析与构建.为此,本文提出了一种非约束环境下的轻量级多人三维动作捕捉系统,该系统便于组装,仅需少量的(4～6个)普通相机或摄像头,并且适用于一般的室内外环境,对场景中的背景和人物对象的着装没有特殊要求.首先,为了提升三维人体动作捕捉的精度,设计并实现了一个鲁棒的基于多视图的多人三维姿态估计算法,它充分利用了多视角的信息互补,并引入了一个跨帧连贯性约束来过滤异常的二维人体关节点检测结果以抑制其对三维人体姿态估计的影响.其次,围绕多视图数据的实时采集与传输及其有效处理,构建了一套轻量级的硬件系统,同时提出了一种数据流向可控的模块化并行处理机制,最终实现了一个灵活、易维护扩充且高效的软硬件系统框架.最后,基于多个公开数据集与自采数据集,通过大量实验,充分验证了本文所提出算法与系统的准确性、高效性和鲁棒性.
Abstract
Markerless multi-person 3D motion capture techniques find applications in various domains such as digital film and animation,3D games,virtual-real interaction,and human gait analysis.However,most research papers tend to focus on algorithm innovation rather than building an effective motion capture system.In this paper,we present a lightweight motion capture system that uses only sparse multiview cameras for multi-person interactive scenarios.Our algorithm achieves accurate 3D human pose estimation by fully leveraging the complementary information from multiple views and filtering unexpected 2D human joint detections.We set up a lightweight hardware system that enables real-time acquisition,transfer,and effective processing of multiview data.Furthermore,we introduce a modularized parallel processing scheme that facilitates dataflow control.Finally,we achieve a scalable,maintainable,and efficient framework for hardware and software configurations.The experimental results demonstrate the accuracy,effectiveness,and efficiency of our algorithms and the proposed lightweight motion capture system.
关键词
动作捕捉/非接触式动捕/实时动捕/多人三维姿态估计/跨视角姿态关联Key words
motion capture/markerless motion capture/realtime motion capture/multi-person 3D pose estimation/cross-view pose association引用本文复制引用
基金项目
浙江省自然科学基金(LY21F020010)
国家重点研发计划(2018YFB1404102)
出版年
2023
NETL
NSTL
万方数据