基于迁移学习双阶段训练的情感语音克隆技术
Emotional voice cloning technology based on two-stage training of transfer learning
李囡 1郭浩 1相洁1
作者信息
- 1. 太原理工大学信息与计算机学院,山西晋中 030600
- 折叠
摘要
为解决传统基于SV2TTS架构的语音克隆系统合成语音缺乏丰富表达能力的问题,提出一种基于迁移学习双阶段训练说话人编码器的情感语音克隆方法.在说话人识别技术基础上,利用迁移学习,对说话人编码器进行音色克隆训练;在情感克隆训练阶段,采用情感语音数据集对其参数进行微调,提取具有情感信息的说话人特征.将此特征作为合成器的输入对梅尔谱的生成过程进行调节,通过声码器将此梅尔谱转换为具有目标说话人信息的情感语音.主客观分析结果表明,该方法所生成的克隆语音的情感相似度更高,音色相似度也有一定改善.
Abstract
To solve the problem that the traditional voice cloning system based on SV2TTS lacks rich expression ability,an emo-tional voice cloning method based on transfer learning two-stage training speaker encoder was proposed.Based on the speaker recognition technology,the transfer learning was used to clone the voice of the speaker encoder.In the emotion cloning training phase,the emotion speech data set was used to fine-tune the encoder parameters,so that it could extract speaker features with emotion information.This feature was used as the input of the synthesizer to adjust the generation process of the Mel spectrum.The Mel spectrum was converted into emotional speech with target speaker information through a vocoder.The results show that the emotional similarity of cloned speech generated using this method is higher,and the speaker similarity is also improved in both objective and subjective evaluation.
关键词
迁移学习/情感语音克隆/情感语音合成/双阶段训练/说话人识别/说话人编码/参数微调Key words
transfer learning/emotional voice cloning/emotional speech synthesis/two-stage training/speaker recognition/speaker encoding/fine-tuning引用本文复制引用
基金项目
国家自然科学基金(61876124)
国家自然科学基金(61873178)
山西省科技厅基础研究基金(20210302123129)
山西省科技厅基础研究基金(20210302124166)
山西省科技厅基础研究基金(20210302123099)
出版年
2024
NETL
NSTL
维普
万方数据