首页|一种街景影像的多模态地理场所情感度量方法

一种街景影像的多模态地理场所情感度量方法

扫码查看
基于视觉特征的感知是居民认知城市环境的重要方式,街景影像作为一种从人类视觉角度反映城市环境的泛在地理信息数据,近些年被广泛用于典型场所分类及空间特征分析的研究中.已有研究表明利用街景影像所提取的视觉特征能够预测地理场所的情感,但现有方法未能充分利用影像内的丰富语义信息.针对城市地理场所的情感度量问题,试图利用语义分割和自然语言处理的方法,将街景影像地理实体实例嵌入为高维数值特征向量,与视觉模态特征构成多模态特征,以提升地理场所情感度量的精度.实验结果表明,相比于现有方法,所提出的多模态特征可提升街景影像的表征能力,进而获得更高的场所情感度量精度;所提出的方法对深度挖掘街景影像信息,辅助理解场所情感分类结果有一定的意义.
A Multi-model Learning Method for Geographical Emotion Detection Based on Street View Images
As an important way to sense the urban living environment,vision has been playing a vital role in dis-covering the urban physical environment.As a type of ubiquitous geospatial data,the street view images describe the urban environment from the perspective of visual perception,which makes it popular in typical place classifica-tion and spatial characteristic analysis researches.In previous researches,the ability of street view images has been proved in measuring human perceptions,but the rich contexts in images have been hardly fully utilized.A learning method using the semantic segmentation and natural language processing is proposed for measuring human emotion perceptions.By embedding geographical entities into the word vector and concatenating with visual feature vector,fusion features show better performance in the classification task.The results indicate that the proposed method learns meaningful representations comparing to existing methods,which could significantly improve performance in the emotion prediction task.And the information from street images can be fully extracted,which contributes to a better understanding of the underlying urban environments.

street view imagesdeep learningsemantic segmentationgeospatial big datanatural language pro-cessing

陈欣、郭文月、孙群、李少梅、温伯威

展开 >

信息工程大学,河南 郑州 450001

街景影像 深度学习 语义分割 地理大数据 自然语言处理

2024

测绘科学技术学报
信息工程大学科研部

测绘科学技术学报

影响因子:0.594
ISSN:1673-6338
年,卷(期):2024.40(6)