首页|融合多尺度特征的多模态情感分析模型设计与实验

融合多尺度特征的多模态情感分析模型设计与实验

扫码查看
多模态情感分析的关键是情感特征提取.针对现有研究存在特征表达能力不强、情感分析效率较低等问题,研究设计融合多尺度特征的多模态情感分析模型,以实现高效、多模态情感分析.对音频、图像和文本等多模态数据向量化;设计多尺度卷积神经网络并特征提取;进行统计池化处理,得到特征向量的标准差、最大值和平均值,引入注意力机制特征向量融合;利用跨模态数据及特征向量情感倾向识别.自建实验数据集比较结果表明,在单模态情感分析和多模态情感分析中,所提模型较现有模型均具有更高准确率,多模态数据的统计学特征以及统计学加权特征对于多模态情感分析具有重要价值.
Design and Experiment of Multi-Modal Sentiment Analysis Model by Fusing Multi-scale Features
The research of sentiment feature extraction exists some challenges,such as poor characterization of extracted feature and low efficiency of sentiment analysis.In view of this,this paper aims to design the multi-modal sentiment analysis model by fusing multi-scale features(MSAM),to efficiently fulfill the multi-modal sentiment analysis task.In this model,the multi-modal data are represented as vector.Then,multi-scale convolutional neural network(MSCNN)is used to feature extraction.The average,maximum and minimum of the extracted feature can be obtained after statistical pooling.The features are fused with the help of the attention mechanism.Finally,it can be predicted the sentiment tendency based on the multi-modal data and the extracted features.The comparative experimental results show that the model proposed in this paper performs better than several traditional models in the mono-modal and multi-modal situations.Specially,the statistical feature and the weighted statistical feature play an important role in multi-modal sentiment analysis.

multi-modalsentiment analysismulti-scale featurestatistical featurefeature fusion

刘忠宝、雷宇飞

展开 >

山东外国语职业技术大学信息工程学院,山东日照 276826

北京语言大学信息科学学院,北京 100083

泉州信息工程学院软件学院,福建泉州 362000

多模态 情感分析 多尺度特征 统计学特征 特征融合

福建省社会科学基金项目山东省社科联2024年度人文社会科学课题

FJ2022A01824BJX102

2024

实验室研究与探索
上海交通大学

实验室研究与探索

CSTPCD北大核心
影响因子:1.69
ISSN:1006-7167
年,卷(期):2024.43(9)