Deep learning-based activity recognition and fine motor identification using 2D skeletons of cynomolgus monkeys

Chuxi Li ¹Zifan Xiao ²Yerong Li ¹Zhinan Chen ¹Xun Ji ³Yiqun Liu ⁴Shufei Feng ⁵Zhen Zhang ⁵Kaiming Zhang ⁶Jianfeng Feng ²Trevor W.Robbins ⁷Shisheng Xiong ¹Yongchang Chen ⁵Xiao Xiao²

扫码查看

作者信息

1. School of Information Science and Technology Micro Nano System Center,Fudan University,Shanghai 200433,China
2. Department of Anesthesiology,Huashan Hospital;Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence,Ministry of Education;Behavioral and Cognitive Neuroscience Center,Institute of Science and Technology for Brain-Inspired Intelligence,MOE Frontiers Center for Brain Science,Fudan University,Shanghai 200433,China
3. Kuang Yarning Honors School,Nanjing University,Nanjing,Jiangsu 210023,China
4. Shanghai Key Laboratory of Intelligent Information Processing,School of Computer Science,Fudan University,Shanghai 200433,China
5. State Key Laboratory of Primate Biomedical Research;Institute of Primate Translational Medicine,Kunming University of Science and Technology,Kunming,Yunnan 650500,China
6. New Vision World LLC.,Aliso Viejo,California 92656,USA
7. Department of Anesthesiology,Huashan Hospital;Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence,Ministry of Education;Behavioral and Cognitive Neuroscience Center,Institute of Science and Technology for Brain-Inspired Intelligence,MOE Frontiers Center for Brain Science,Fudan University,Shanghai 200433,China;Behavioural and Clinical Neuroscience Institute,University of Cambridge,Cambridge,CB2 1TN,UK
折叠

Abstract

Video-based action recognition is becoming a vital tool in clinical research and neuroscientific study for disorder detection and prediction.However,action recognition currently used in non-human primate(NHP)research relies heavily on intense manual labor and lacks standardized assessment.In this work,we established two standard benchmark datasets of NHPs in the laboratory:MonkeyinLab(MiL),which includes 13 categories of actions and postures,and MiL2D,which includes sequences of two-dimensional(2D)skeleton features.Furthermore,based on recent methodological advances in deep learning and skeleton visualization,we introduced the MonkeyMonitorKit(MonKit)toolbox for automatic action recognition,posture estimation,and identification of fine motor activity in monkeys.Using the datasets and MonKit,we evaluated the daily behaviors of wild-type cynomolgus monkeys within their home cages and experimental environments and compared these observations with the behaviors exhibited by cynomolgus monkeys possessing mutations in the MECP2 gene as a disease model of Rett syndrome(RTT).MonKit was used to assess motor function,stereotyped behaviors,and depressive phenotypes,with the outcomes compared with human manual detection.MonKit established consistent criteria for identifying behavior in NHPs with high accuracy and efficiency,thus providing a novel and comprehensive tool for assessing phenotypic behavior in monkeys.

Key words

Action recognition/Fine motor identification/Two-stream deep model/2D skeleton/Non-human primates/Rett syndrome

引用本文复制引用

基金项目

National Key R&D Program of China(2021ZD0202805)

National Key R&D Program of China(2019YFA0709504)

National Key R&D Program of China(2021ZD0200900)

National Defense Science and Technology Innovation Special Zone Spark Project(20-163-00-TS-009-152-01)

National Natural Science Foundation of China(31900719)

National Natural Science Foundation of China(U20A20227)

National Natural Science Foundation of China(82125008)

Innovative Research Team of Highlevel Local Universities in Shanghai()

Science and Technology Committee Rising-Star Program(19QA1401400)

111 Project(B18015)

Shanghai Municipal Science and Technology Major Project(2018SHZDZX01)

Shanghai Center for Brain Science and Brain-Inspired Technology()

出版年

2023

动物学研究

中国科学院昆明动物研究所中国动物学会

动物学研究

CSTPCDCSCD

影响因子：0.582

ISSN：0254-5853

参考文献量1

段落导航