A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps

扫码查看

原文链接

NETL
NSTL
万方数据

外文摘要：In this work,we present a reconfigurable data glove design to capture different modes of human hand-object interactions,which are critical in training embodied artificial intelligence(Al)agents for fine manipulation tasks.To achieve various downstream tasks with distinct features,our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time.In the tactile-sensing mode,the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material;this design minimizes interference during complex hand movements.The virtual reality(VR)mode enables real-time interaction in a physically plausible fashion:A caging-based approach is devised to determine stable grasps by detecting collision events.Leveraging a state-of-the-art finite element method,the simulation mode collects data on fine-grained four-dimensional manipulation events comprising hand and object motions in three-dimensional space and how the object's physical properties(e.g.,stress and energy)change in accordance with manipulation over time.Notably,the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions.In a series of experiments,we characterize our data glove in terms of individual sensors and the overall sys-tem.More specifically,we evaluate the system's three modes by ① recording hand gestures and associ-ated forces,② improving manipulation fluency in VR,and ③ producing realistic simulation effects of various tool uses,respectively.Based on these three modes,our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments,thereby opening up new avenues for the learning of manipulation skills for embodied AI agents.

外文关键词：

Data gloveTactile sensingVirtual realityPhysics-based simulation

作者：

Hangxin Liu、Zeyu Zhang、Ziyuan Jiao、Zhenliang Zhang、Minchen Li、Chenfanfu Jiang、Yixin Zhu、Song-Chun Zhu

展开 >

作者单位：

National Key Laboratory of General Artificial Intelligence,Beijing Institute for General Artificial Intelligence(BIGAI),Beijing 100080,China

Center for Vision,Cognition,Learning,and Autonomy,University of California,Los Angeles,CA 90095,USA

Multi-Physics Lagrangian-Eulerian Simulations Laboratory,Department of Mathematics,University of California,Los Angeles,CA 90095,USA

Institute for Artificial Intelligence,Peking University,Beijing 100871,China

Department of Automation,Tsinghua University,Beijing 100084,China

展开 >

基金：

国家重点研发计划Beijing Nova Program

项目编号：

2021ZD0150200

出版年：

2024

DOI：

10.1016/j.eng.2023.01.009

工程(英文)

CSTPCDEI

ISSN：2095-8099

年,卷(期)：2024.32(1)

参考文献量102