多模态情感分析作为情感分析中的一个新兴的研究领域,旨在利用多模态信号识别情感.已有的研究方法在缓解不同模态的分布差异和模型的鲁棒性方面存在不足.为了解决以上问题,该文提出了一种基于二元模态表示学习与融合网络(Bimodal Representation Learning and Fusion Network,BRLAFN).具体而言,该网络学习二元模态之间的模态共同特征来减少模态差距,学习模态私有特征,确保模态特定信息的存在.然后,采用重建损失和循环一致性损失来确保模态特征的稳定性和一致性.最后,利用跨模态注意力实现模态间的互补学习.该文在流行情感分析基准(CMU-MOSI和CMU-MOSEI)上的实验结果表明,与同期最先进的模型相比,该网络取得了显著的性能提升.
Multimodal Sentiment Analysis based on Bimodal Representation Learning and Fusion Network
Multimodal sentiment analysis,as an emerging research field in sentiment analysis,aims to use multimo-dal signals to identify emotions.This paper proposes a network based on Bimodal Representation Learning and Fu-sion(BRLAFN).Specifically,the network learns common features between two modalities to reduce modal gaps,and learns modality-unique features to ensure the existence of modality-specific information.Then,the reconstruction loss and cycle consistency loss are adopted to ensure the stability and consistency of the modal fea-tures.Finally,cross-modal attention is used to achieve complementary learning between modalities.The experimen-tal results on popular sentiment analysis benchmarks(CMU-MOSI and CMU-MOSEI)show that the network has achieved significant improvements compared to state-of-the-art models.
NETL
NSTL
万方数据
