Multimodal Psychological Evaluation Algorithm for Power Grid Industry
To solve the intelligent psychological evaluation task for high-risk industry employees in the power grid before work,a multimodal psychological evaluation algorithm is proposed for employees in the power grid industry,which includes expressions,sounds,and walking pos-ture.Firstly,construct a dataset of employees in the power grid industry,extracting facial RGB image sequences,audio ComparE feature sets,and human skeleton keypoint sequences from videos;Secondly,residual networks and bidirectional long short-term memory networks are used to extract facial visual features,audio features are extracted in time windows,and gait features are extracted in spatiotemporal graph convolutional networks,respectively,to obtain the optimal single modal models;Finally,a deep learning training method for polarity loss function and a multimodal fusion algorithm based on attention mechanism are proposed to obtain the optimal multimodal psychological state evaluation model by fusing the output features of a single modal model.The experiment shows that multimodal fusion can significantly improve the accuracy of psychological evaluation compared with single-mode system,and the accuracy rate of the four classification tasks of psycholog-ical labels reaches 65.66%.Compared with the model based on facial expression,voice,and gait,the effect of multimodal fusion is increased by 18.04%,21.22%,and 13.28%respectively.
multimodalitydeep learningattention mechanismpsychological evaluationpower grid industry
万方数据
维普
