基于知识蒸馏的跨模态语音情感分类
Cross-modal Speech Sentiment Classification Based on Knowledge Distillation
尤佩雯 1王晶晶 1高晓雅 1李寿山1
作者信息
- 1. 苏州大学 自然语言处理实验室 计算机科学与技术学院,江苏 苏州 215006
- 折叠
摘要
针对语音情感分类任务面临的语音数据标注困难的挑战,该文提出了一种新的跨模态语音情感分类任务,其可以使用文本模态数据(源端)帮助语音模态数据(目标端)进行情感分类.在此基础上,提出了一种基于知识蒸馏的跨模态情感分类模型,旨在通过知识蒸馏方法将文本情感分类模型(教师模型)学习到的预训练先验知识蒸馏到语音情感分类模型(学生模型)中.该模型的特色在于无须在测试端依赖昂贵的语音识别技术,可对原始语音数据直接进行情感分类,有利于该模型在实际语音情感分类应用场景中大规模落地.实验结果表明,该文所提出的方法可以有效利用文本模态分类的经验来提升语音模态的分类效果.
Abstract
This paper proposes a new cross-modal speech sentiment classification task,which aims to leverage the text modal data as the source side to classify the speech modal data on the target side.This paper designs a cross-modal sentiment classification model based on knowledge distillation,which is intended to distill the prior pre-train-ing knowledge learning from the text-modal sentiment classification model(teacher model)into the speech-modal sentiment classification model(student model).The proposed model is distinguished by that its capability of direct analysis of the original speech data without relying on the speech recognition technology,which is crucial to large-scale implementation in the actual speech emotion analysis application scenarios.Experimental results show that the proposed method can effectively use the experience of text modal sentiment classification to improve the effect of speech modal sentiment classification.
关键词
跨模态/知识蒸馏/情感分类Key words
cross-modal/knowledge distillation/sentiment classification引用本文复制引用
基金项目
国家自然科学基金(62006166)
国家自然科学基金(62076175)
国家自然科学基金(62076176)
中国博士后科学基金(2019M661930)
江苏高校优势学科建设工程资助项目()
出版年
2024
NETL
NSTL
万方数据