首页|基于抽象关系场景图的图像情感识别

基于抽象关系场景图的图像情感识别

扫码查看
原文链接
  • 国家科技期刊平台
  • NETL
  • NSTL
  • 万方数据
图像情感识别是通过分析视觉刺激来预测人类情感的抽象过程.现有方法大多缺乏对对象间关系以及对象与场景间相互作用的关注,并且对象间复杂多样的关系难以得到充分利用,进而导致难以正确对图像情感进行预测.为解决上述问题,提出一种基于抽象关系场景图的图像情感识别方法.首先,构建对象和属性检测器来提取图像中对象及其属性的特征.其次,使用对象特征推理对象间的亲密度和抽象关系特征,进而构建抽象关系场景图.再次,提出抽象关系图卷积网络来推理抽象关系场景图.最后,设计渐进式注意力机制对多个对象特征进行融合,以得到图像的整体对象特征.在FI、EmotionRoI和Twitter I公开数据集上的试验结果表明,该方法的分类准确率优于现有方法.
Image sentiment recognition based on the abstract relational scene graph network
Image sentiment recognition is an abstract process of forecasting human emotions by analysis of various visu-al stimuli.Most of the earlier literature does not focus on the relationships among objects and the interactions between objects and scenes,and the complex and diverse relationships among objects are difficult to fully exploit,resulting in difficulty in correctly forecasting image sentiment.To deal with this problem,we develop an abstract relational scene graph network for image sentiment recognition.First,an object and attribute detector is generated to extract object fea-tures and their corresponding attribute features from images.Second,the affinities and abstract relationship features among objects are inferred through object features,and then the abstract relational scene graph is generated.Moreover,an abstract relational graph convolutional network is developed for reasoning the abstract relational scene graph.Last,a progressive attention mechanism is designed to fuse multiple object features to acquire the overall object feature of the image.Application on three public datasets,FI,EmotionRoI,and Twitter I,demonstrates that the classification accuracy of the proposed method is better than that of the existing methods.

image sentiment recognitionabstract relationshipscene graphgraph convolutional networkattention mechanismconvolutional neural networkvisual sentiment analysisdeep learning

康博、钱艺、文益民

展开 >

桂林电子科技大学 广西图像图形与智能处理重点实验室, 广西 桂林 541004

图像情感识别 抽象关系 场景图 图卷积网络 注意力机制 卷积神经网络 视觉情感分析 深度学习

国家自然科学基金广西壮族自治区重点研发计划广西图像图形与智能处理重点实验室项目

62366011桂科AB21220023GIIP 2306

2024

10.11992/tis.202303009

智能系统学报
中国人工智能学会　哈尔滨工程大学

智能系统学报

CSTPCD北大核心
影响因子：0.672
ISSN：1673-4785
年,卷(期)：2024.19(2)
  • 37