首页|复谱映射下融合高效Transformer的语音增强方法

复谱映射下融合高效Transformer的语音增强方法

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
针对卷积神经网络(Convolutional Neural Network,CNN)过去在语音增强中表现优异但对全局特征捕获不足,以及Transformer近年展现出长序列间依赖优势但又存在局部细节特征丢失、参数量大等问题,该文为了充分利用CNN与Transformer的优势并弥补各自不足,提出了一种在复频谱映射下的新型卷积模块与高效Transformer融合的单通道语音增强网络.该网络由编码层、传输层与双分支解码层组成:在编解码部分设计了一种协作学习模块(Collaborative Learning Block,CLB)来监督交互信息,在减少参数量的同时提高主干网络对复特征的获取能力;传输层中则提出一种时频空间注意Transformer模块分别对语音子频带和全频带信息建模,充分利用声学特性来模拟局部频谱模式并捕获谐波间依赖关系.将该模块进一步与通道注意分支相结合,设计了一种可学习的双分支注意融合(Dual-branch Attention Fusion,DAF)机制,从空间-通道角度提取上下文特征以加强信息的多维度传输;最后,在此基础上搭建一种高斯加权渐进网络作为中间传输层,通过堆叠DAF模块进行加权求和后输出以充分利用深层特征,使得解码过程更具鲁棒性.分别在英文VoiceBank-DEMAND数据集、中文THCHS30语料库与115种环境噪声下进行消融以及综合对比实验,结果表明,该文方法仅以最小0.68×106的参数量,相比于大部分最新相关网络模型取得了更优的主、客观指标,具有较为突出的增强性能与泛化能力.
Speech Enhancement Method Based on Complex Spectrum Mapping with Efficient Transformer
Convolution neural networks(CNNs)perform well in speech enhancement but fail to capture global features.In addition,in recent years,a transformer has shown the advantage of long sequence dependence,but there have been problems such as local detail feature loss and a large number of parameters.In order to make full use of the advantages of a CNN and transformer and make up for their shortcomings,this study investigated a single channel speech enhance-ment network based on the fusion of novel convolutional modules and an efficient transformer under complex spectrum mapping.The network was composed of a coding layer,transmission layer,and double-branch decoding layer.In the codec part,a collaborative learning block was designed to supervise the interactive information,which reduced the num-ber of parameters and improved the ability of the backbone network to obtain complex features.In the transmission layer,a time-frequency spatial attention transformer module was proposed to model the voice sub-band and full band in-formation,and made full use of acoustic characteristics to simulate local spectrum patterns and capture the harmonic de-pendency.By combining this module with a channel attention branch,a learnable dual-branch attention fusion(DAF)mechanism was designed to extract contextual features from the space-channel perspective to enhance multi-dimensional information transmission.Finally,a Gaussian weighted asymptotic network was built as the intermediate transmission layer,and the weighted sum output was performed by stacking DAF modules to make full use of the deep features and make the decoding process more robust.Ablation and comprehensive comparison experiments were conducted on the English VoiceBank-DEMAND dataset,the Chinese THCHS30 corpus,and 115 kinds of ambient noise.The results showed that the proposed method achieved better subjective and objective indicator results with the minimum number of parameters(0.68×106)compared with most of the latest correlation network models.It demonstrated outstanding en-hancement performance and generalization ability.

speech enhancementcomplex spectral mappingefficient transformerlightweight network

张天骐、罗庆予、张慧芝、方蓉

展开 >

重庆邮电大学通信与信息工程学院,重庆 400065

语音增强 复频谱映射 高效Transformer 轻量型网络

国家自然科学基金国家自然科学基金国家自然科学基金国家自然科学基金重庆市自然科学基金信号与信息处理重庆市市级重点实验室建设项目重庆市教育委员会科研项目重庆市教育委员会科研项目

61671095617020656170106761771085cstc2021jcyjmsxmX0836CSTC2009CA2003KJQN201900601KJ1600429

2024

10.16798/j.issn.1003-0530.2024.02.018

信号处理
中国电子学会

信号处理

CSTPCD北大核心
影响因子：1.502
ISSN：1003-0530
年,卷(期)：2024.40(2)
  • 22