Night behavior recognition based on multi-mode feature fusion
Action recognition in low light scenes such as night is of great significance to the fields of security,automatic driving and so on.Aiming at the problems of poor recognition effect and poor robustness of existing methods in low-light environment,a multimodal aggregate low light environment action recognition method based on feature guidance is proposed.Firstly,the hierarchical skeleton fusion network for illumination enhancement is designed.The illumination enhancement algorithm and hierarchical spatiotemporal feature fusion strategy are used to obtain the action features that focus on the expression of human behavior itself,and improve the accuracy degradation caused by skeleton missing in low-light scenes.Secondly,the apparent feature extraction network based on a fusion spatiotemporal conversion module is designed.Spatiotemporal features containing rich scene information on a 2D feature extraction network are efficiently captured using a zero-parameter temporal displacement module.Then,a multi-modal aggregation network based on feature guidance is designed.The feature guidance strategy is used to perform the deep information interaction between skeleton features and RGB features,so as to realize the comprehensive characterization of behavior features.Finally,the full connection layer is used for feature classification to complete behavior recognition.The experimental results show that the proposed method can be well applied to human action recognition tasks in low light environment.
NETL
NSTL
万方数据
