首页|联合语义提示和记忆增强的弱监督跳绳视频异常检测方法

联合语义提示和记忆增强的弱监督跳绳视频异常检测方法

扫码查看
原文链接
  • 国家科技期刊平台
  • NETL
  • NSTL
  • 万方数据
面向全国的学生综合评价与发展平台采集了百万级学生跳绳运动数据用于中小学生身体素质测评，然而在采集的跳绳视频中存在着非跳绳视频、人物未全身出镜等不符合拍摄要求的各类异常视频，严重影响了后续学生身体素质测评模型的准确性和鲁棒性。针对上述问题，提出一种联合语义提示和记忆增强的弱监督跳绳视频异常检测方法。该方法首先提取正常跳绳视频和异常跳绳视频的视觉特征，将正常特征和异常特征成对训练，增强模型对异常视频特征的感知能力;其次设计两个自监督记忆网络分别存储和分离正常视频和异常视频的特征，进一步增强模型的特征表达能力;最后引入提示学习方法迁移大规模预训练模型中的多种跳绳异常类型的语义先验知识，增强模型在样本不足的情况下对多种异常类型语义信息的理解。实验结果表明，该方法在自建的跳绳异常检测数据集(SRAD)上的AUC为94。14%，相较于基准方法提升了2。71个百分点，具有较高的准确性。该方法对实现身体素质的智能测评、推动教育评价改革具有重要意义。
Weakly Supervised Video Anomaly Detection Method for Rope Skipping Combined with Semantic Prompts and Memory Enhancement
The student evaluation enhancement platform collects millions of jump rope movement data points from primary and secondary school students for physical fitness assessment.However,in the collected jump rope videos,there are various abnormal videos that do not meet the shooting requirements,such as non-jump rope videos,and characters that do not appear in full body.This seriously affects the accuracy and robustness of the follow-up physical fitness assessment model of students.To solve these problems,this study proposes a weakly supervised video anomaly detection method that combines semantic cues and memory enhancement.First,the visual features of normal and abnormal skipping rope videos are extracted,and the normal and abnormal features are trained in pairs to enhance the model's perception of abnormal video features.Second,two self-supervised memory networks are designed to store and separate the features of normal and abnormal videos to further enhance the feature representation ability of the model.Finally,a prompt learning method is introduced to transfer the semantic prior knowledge of various skip rope exception types in a large-scale pre-training model to enhance the model's understanding of the semantic information of various exception types in the case of insufficient samples.The experimental results show that the AUC of the proposed method on the self-built Skip Rope Anomaly Detection(SRAD)dataset is 94.14%,which is 2.71 percentage points higher than that of the benchmark method,thereby exhibiting high accuracy.The proposed method is of great significance for realizing intelligent physical fitness evaluation and promoting educational assessment reform.

video anomaly detectionprompt learningmemory networkweakly supervised learningphysical fitness assessment

周炫余、吴莲华、郑勤华、肖天星、王紫璇、张思敏

展开 >

湖南师范大学基础教育大数据研究与应用重点实验室,湖南 长沙 410006

北京师范大学远程教育中心,北京 100091

视频异常检测 提示学习 记忆网络 弱监督学习 身体素质测评

国家重点研发计划国家自然科学基金国家自然科学基金湖南省自然科学基金湖南省自然科学基金

2021YFC334080062007007617031552023JJ304152022JJ30395

2024

10.19678/j.issn.1000-3428.0069415

计算机工程
华东计算技术研究所　上海市计算机学会

计算机工程

CSTPCD北大核心
影响因子：0.581
ISSN：1000-3428
年,卷(期)：2024.50(7)