首页|超越单一模态:多模态计算传播研究的进展与前瞻

超越单一模态:多模态计算传播研究的进展与前瞻

扫码查看
随着计算方法的快速发展,超越文本的视觉计算传播研究开始兴起,但是仍主要体现为单一模态研究(只是从文本转向视觉).本文强调,在媒体内容消费多模态化的时代背景下,我们应当高度重视多模态计算传播研究的发展.通过对SSCI、A&HCI等核心数据库中的31篇多模态计算传播研究的内容分析,我们梳理了现有研究概况、理论概念、研究模式与操作方法.研究发现:多模态计算传播研究发展较晚,亟待加强;研究模式呈现多模态关系(独立/组合)×研究目的(描述/解释)的四个分类;数据主要来自图文社交平台;基于多模态特征生成的组合变量主要包括图文一致性、图文主题、叙事策略、攻击性指数等四种.本文最后从应用生成式人工智能、推进理论建设、促进跨平台跨地区发展等方面,对多模态计算传播研究的未来发展提出建议.
Beyond Unimodality:Advances and Prospects in Multimodal Computational Communication Research
With the rapid evolution of computational methods,there is a noticeable surge in Computational Communication Research(CCR)that transcends traditional text-based communication by delving into the visual aspects.However,the current focus remains predominantly unimodal,merely transitioning from text to images.Confronted with the inherently multimodal nature of media consumption,this article underscores the imperative of prioritizing research in multimodal communication.This study presents a comprehensive review of 31 multimodal computational communication research papers published in SSCI and A&HCI journals.The key findings are as follows:(1)Multimodal computational communication research is a relatively nascent field that demands further development;(2)Research patterns can be categorized into four groups based on the relationship between modalities(independent/combinational)and research objectives(descriptive/explanatory);(3)The primary source of research data is image-text-based social media;and(4)Variables generated through the combination of multimodal features include image-text congruence,image-text themes,image-text narratives,and aggressiveness.The article concludes with actionable suggestions,urging the application of generative AI,the advancement of theory construction,and the promotion of cross-platform and cross-regional development in the realm of multimodal computational communication research.

multimodalcomputational communicationcomputer visioncomputer audition

周葆华、吴雨晴

展开 >

复旦大学 新闻学院,上海 200433

多模态 计算传播 计算机视觉 计算机听觉

国家社科基金重大项目国家社科基金人才项目复旦大学文科先导和创新团队项目复旦大学新闻学院科研创新项目(2023-2024)

20ZDA06022VRC186IDH3353070

2024

传媒观察
新华日报报业集团

传媒观察

CSSCI
影响因子:0.241
ISSN:1672-3406
年,卷(期):2024.481(1)
  • 48