符号学视角下的远程视觉示意图:来自两个多模态语料库的洞见
Semiotically-grounded Distant Viewing of Diagrams:Insights from Two Multimodal Corpora
图奥默·哈帕拉 1约翰·贝特曼 2王振华 3吴启竞 4杨林秀5
作者信息
- 1. 芬兰赫尔辛基大学语言系,芬兰赫尔辛基
- 2. 德国不莱梅大学语言学系、英语系,德国不莱梅
- 3. 上海交通大学马丁适用语言学研究中心,上海 200240
- 4. 大连理工大学外国语学院,辽宁大连 116024
- 5. 山西大学外国语学院,太原 037008
- 折叠
摘要
多模态研究探索人类如何依赖各种有机结合的表达资源进行交流,已建构为一门新兴学科.本文认为,当下数字人文的研究结合多模态研究的洞察力对可视材料进行大规模分析是有益的举措.多模态研究开发的概念为大型语料库和数据集中的各种表意模态提供了合适的元数据方案.本文使用常见的表意模态和示意图来阐释文中提出的方法,并运用统计和计算的方法分析两个新建的多模态语料库.研究结果表明,由多模态驱动的元数据方案可以为大型语料库和数据集的计算分析提供坚实基础.即使是语料库或数据集的设计不支持全面分析多模态交际,多模态理论依然可以用于对各种视觉材料进行比较严格的分析.
Abstract
In this article,we argue for the benefits of combining large-scale analyses of visual materials currently pursued within digital humanities with insights from multimodality research,which is an emerging discipline that studies how human communication relies on appropriate combinations of expressive resources.We show that concepts developed within the field of multimodality research provide appropriate metadata schemes for various modes of expression in large corpora and datasets.We illustrate the proposed approach using a common mode of expression,diagrams,and analyse two recent multimodal diagram corpora using statistical and computational methods.Our results suggest that multimodally-motivated metadata schemes can provide a robust foundation for computational analyses of large corpora and datasets.Even if a corpus or dataset is not designed to support fullblown analyses of multimodal communication,our results imply that multimodality theory can still be used to impose tighter analytical control over a variety of visual materials.
关键词
多模态语料库/远程视觉图表/洞见/符号学/元数据方案Key words
multimodal corpora/distant viewing of diagrams/insights/semiotics/meta-data schemes引用本文复制引用
基金项目
上海交通大学文科创新团队培育计划项目(WKCX007)
教育部人文社会科学研究青年基金项目(22YJC740085)
出版年
2024
NETL
NSTL
万方数据