计算机应用研究2025,Vol.42Issue(1) :100-105.DOI:10.19734/j.issn.1001-3695.2024.05.0187

基于自适应融合技术的多模态实体对齐模型

Multi-modal entity alignment model based on adaptive fusion technology

任楚岚 于振坤 关超 井立志
计算机应用研究2025,Vol.42Issue(1) :100-105.DOI:10.19734/j.issn.1001-3695.2024.05.0187

基于自适应融合技术的多模态实体对齐模型

Multi-modal entity alignment model based on adaptive fusion technology

任楚岚 1于振坤 1关超 1井立志1
扫码查看

作者信息

  • 1. 沈阳化工大学计算机科学与技术学院,沈阳 110142;辽宁省化工过程工业智能化技术重点实验室,沈阳 110142
  • 折叠

摘要

多模态实体对齐旨在识别由结构三元组和与实体相关的图像组成的不同的多模态知识图谱之间的等价实体.现有的多模态实体对齐的研究主要集中在多模态融合策略,忽略了模态缺失和不同模态难以融合的问题,未能充分利用多模态信息.为了解决上述问题,提出了 MACEA模型,该模型使用多模态变分自编码方法主动补全缺失的模态信息,动态模态融合方法整合不同模态的信息并相互补充,模态间对比学习方法对模态间进行建模,这些方法有效解决了模态缺失与模态难以融合的问题.相比于当前基线模型,MACEA的hits@1和MRR指标分别提升了 5.72%和6.78%,实验结果表明,该方法可以有效地识别出对齐实体对,具有较高的准确性和实用性.

Abstract

Multi-modal entity alignment aims to identify equivalent entities between different multi-modal knowledge graphs composed of structured triples and images associated with entities.The existing research on multi-modal entity alignment main-ly focuses on multi-modal fusion strategies,ignoring the problems of modal imbalance and difficulty in integrating different mo-dalities,and fails to fully utilize multi-modal information.To solve these problems,this paper proposed the MACEA model,this model used the multi-modal variational autoencoder method to actively complete the missing modal information,the dy-namic modal fusion method to integrate and complement the information of different modalities,and the inter-modal contrastive learning method to model the inter-modal relations.These methods effectively solve the problems of modal missing and the dif-ficulty in modal fusion.Compared with the baseline model,MACEA improves the hits@1 and MRR indicators by 5.72%and 6.78%,respectively.The experimental results show that the proposed method can effectively identify aligned entity pairs,with high accuracy and practicality.

关键词

实体对齐/知识图谱/多模态/动态融合/模态缺失

Key words

entity alignment/knowledge graph/multi-modal/dynamic fusion/modality missing

引用本文复制引用

出版年

2025
计算机应用研究
四川省电子计算机应用研究中心

计算机应用研究

北大核心
影响因子:0.93
ISSN:1001-3695
段落导航相关论文