Robust Machine Translation Based on Amplifying Hidden Representation Differences
Contrastive learning,as the the mainstream method in robust machine translation,usually adds noise to the input Token layer or the Embedding layer to expand the sample pool and enrich sample styles.However,after being processed by the Encoder,the differences between noise samples and clean samples in Hidden Representations will be decreases.In this paper,we maintain the dissimilarity between noise samples and clean samples in Hidden Representations by directly adding Gaussian noise to the Encoder's Hidden Representations.On the Decoder side,by jointly training the loss of noise samples and KL divergence loss,the target probability distribution of the noise samples is approximated close to that of the clean samples.In the IWSLT2014 De-En task,the proposed method achieves 0.9 BLEU improvement on the clean test set compared to the R-Drop and SimCut.On the noisy test set,the proposed method achieves 0.82 BLEU and 0.63 BLEU improvements,respectively.Applied to the Speech-to-Text(ST)task,the proposed method brings1.3 BLEU improvement on the MuST-C test set,and 3.0 BLEU improvement on the multi-speaker test set of CoVoST 2.in contrast to ConST system.
NETL
NSTL
万方数据
