首页|说话人感知的交叉注意力说话人提取网络

说话人感知的交叉注意力说话人提取网络

扫码查看
原文链接
  • 国家科技期刊平台
  • NETL
  • NSTL
  • 万方数据
目标说话人提取任务的目标是在一段混合音频中提取特定说话人的语音,任务设置上一般会给一段目标说话人注册音频作为辅助信息.现有的研究工作主要有以下不足:(1)说话人识别的辅助网络无法捕获学习注册音频中的关键信息;(2)缺乏混合音频嵌入和注册音频嵌入的交互学习机制.以上不足导致了现有研究工作在注册音频和目标音频之间存在较大差异时有说话人混淆问题.为了解决该问题,提出说话人感知的交叉注意力说话人提取网络(Speaker-aware Cross Attention Speaker Extraction Network,SACAN).SACAN在说话人识别辅助网络引入基于注意力的说话人聚合模块,有效聚合目标说话人声音特性的关键信息和利用混合音频增强目标说话人嵌入.进一步地,SACAN通过交叉注意力构建交互学习机制促进说话人嵌入与混合音频嵌入融合学习,增强了模型的说话人感知能力.实验结果表明,SACAN相比基准方法在STOI和SI-SDRi分别提高了0.0133、1.0695 dB,并在说话人混淆相关评估和消融实验中验证了不同模块的有效性.
Speaker-Aware Cross Attention Speaker Extraction Network
Target speaker extraction aims to extract the speech of the specific speaker from mixed audio,which usually treats the enrolled audio of the target speaker as auxiliary information.Existing approaches mainly have the following limitations:the auxiliary network for speaker recognition cannot capture the critical information from enrolled audio,and the second one is the lack of an interactive learning mechanism between mixed and enrolled audio embedding.These limitations lead to speaker confusion when the difference between the enrolled and target audio is significant.To address this,a speaker-aware cross-attention speaker extraction network(SACAN)is proposed.First,SACAN introduces an attention-based speaker aggregation module in the speaker recognition auxiliary network,which effectively aggregates critical information about target speaker characteristics.Then,it uses mixed audio to enhance target speaker embedding.After that,to promote the integration of speaker embedding and mixed audio embedding,SACAN builds an interactive learning mechanism through cross-attention and enhances the speaker perception ability of the model.The experimental results show that SACAN improves by 0.0133 and 1.0695 in terms of STOI and SI-SDRi when compared with the benchmark model,validating the effectiveness of the proposed module in speaker confusion assessment and ablation experiments.

speech separationtarget speaker extractionspeaker embeddingcross attentionmulti-task learning

李卓璋、许柏炎、蔡瑞初、郝志峰

展开 >

广东工业大学 计算机学院,广东 广州 510006

汕头大学 理学院,广东 汕头 515063

语音分离 目标说话人提取 说话人嵌入 交叉注意力 多任务学习

科技创新2030-"新一代人工智能"重大项目国家优秀青年科学基金国家自然科学基金国家自然科学基金国家自然科学基金

2021ZD011150162122022618760436197605262206064

2024

10.12052/gdutxb.230037

广东工业大学学报
广东工业大学

广东工业大学学报

影响因子：0.628
ISSN：1007-7162
年,卷(期)：2024.41(3)
  • 33