Image description translation translate image description with the image modal information in an end-to-end system.The traditional image description translation is to assist the translation of the source language by using the vital feature in the image.To capture the source language context that affects the adequacy of the translation to-gether with the target language context that affects the fluency,this paper proposes a multi-modal information fu-sion decoding method based on gating mechanism for the image description translation.Our model uses context gates to dynamically adjusts the contribution of the source and target language contexts to the translation results,improving both the adequacy and fluency of translation results.Experiments show that the method increases the per-formance of image description translation with 1.3％,1.0％,1.5％and 1.4％,respectively,on the four tasks of En-De and En-Fr in Multi30k-16 and Multi30k-17.
关键词
图像描述翻译/多模态机器翻译/上下文门控/忠实度及流畅度
Key words
image description translation/multimodal machine translation/context gates/adequacy and fluency
维普
万方数据