首页|基于多特征融合时差网络的带式输送机区域违规行为识别

基于多特征融合时差网络的带式输送机区域违规行为识别

扫码查看
原文链接
  • 万方数据
  • 维普
现有的煤矿井下带式输送机区域违规行为(如攀爬、跨越、倚靠带式输送机等)识别方法对特征提取不充分、难以考虑到行为时间差异,导致违规行为识别准确率不高.针对该问题,基于ResNet50模型,提出了一种基于多特征融合时差网络(MFFTDN)的带式输送机区域违规行为识别方法,将多特征融合和时间差分进行结合,对不同时间段的行为进行多特征融合.首先在原始模型ResNet50的第2和第3阶段引入短期多特征融合(STMFF)模块,将来自多个连续帧的时间和特征拼接在一起,再对融合后的特征进行时间差分计算,即相邻帧的特征差值,以在短期内捕捉局部动作变化.然后在原始模型ResNet50的第4阶段引入长期多特征融合(LTMFF)模块,将来自连续帧的短期多特征拼接在一起,再对相邻时间点的特征进行时间差分计算,以获取行为的长期多特征.最后将融合后的特征进行分类,输出识别结果.实验结果表明:①该方法的平均精度和准确率较原始模型ResNet50分别提高了 8.18％和8.47％,说明同时引入STMFF和LTMFF模块能够有效提取到不同时间段的多特征信息.②该方法在自建煤矿井下带式输送机区域违规行为数据集上的准确率为89.62％,平均精度为89.30％,模型的参数量为197.2 ×106.③Grad-CAM热力图显示,该方法能够更有效地关注到违规行为的关键区域,精确捕捉到井下带式输送机区域的违规行为.
Recognition of violations in belt conveyor area based on multi-feature fusion for time-difference network
The existing methods for recognizing violations in the underground belt conveyor area of coal mines suffer from insufficient feature extraction and difficulty in considering time differences in behavior.It results in low accuracy in recognizing violations(such as climbing,crossing,leaning,etc.)in the area.In order to solve the above problems,a belt conveyor area violation recognition method based on multi-feature fusion for time-difference network(MFFTDN)is proposed using the ResNet50 model.The method combines multi-feature fusion with time difference to perform multi-feature fusion on behaviors in different time periods.Firstly,the short-term multi-feature fusion(STMFF)module is introduced in the second and third stages of the original ResNet50 model,which concatenates the time and features from multiple consecutive frames together.Then the method performs time-difference calculation on the fused features,that is,the feature difference between adjacent frames,to capture local action changes in a short period of time.Secondly,in the fourth stage of the original ResNet50 model,a long-term multi-feature fusion(LTMFF)module is introduced to concatenate short-term multi-features from consecutive frames,and perform time-difference calculations on features from adjacent time points to obtain long-term multi-feature of behavior.Finally,the method classifies the fused features and outputs the recognition results.The experimental results show the following points.① The average accuracy and precision of the MFFTDN based belt conveyor area violation recognition method have increased by 8.18％and 8.47％respectively compared to the original model ResNet50.It indicates that the simultaneous use of STMFF and LTMFF modules can effectively extract multi-feature information from different time periods.② The accuracy of this recognition method on the self built dataset of violations in the underground belt conveyor area of coal mines is 89.62％,with an average precision of 89.3％and a model parameter size of 197.2× 106.③ The Grad CAM heatmap shows that this recognition method can more effectively focus on key areas of violations and more accurately capture violations in the underground belt area of the mine.

belt conveyor areaunsafe behaviorrecognition of violationsshort-term multi-feature fusionlong-term multi-feature fusionmulti-feature fusion for time difference networktime-difference

马天、姜梅、杨嘉怡、张杰慧、丁旭涵

展开 >

西安科技大学计算机科学与技术学院,陕西西安 710054

西安科技大学安全科学与工程学院,陕西西安 710054

带式输送机 不安全行为 违规行为识别 短期多特征融合 长期多特征融合 多特征融合时差网络 时间差分

国家重点研发计划项目国家自然科学基金项目国家自然科学基金项目陕西省自然科学基础研究计划项目

2021YFB400090562101432621023092022JM-508

2024

10.13272/j.issn.1671-251x.2023080108

工矿自动化
中煤科工集团常州研究院有限公司

工矿自动化

CSTPCD北大核心
影响因子：0.867
ISSN：1671-251X
年,卷(期)：2024.50(7)
  • 9