文本图像翻译旨在将嵌在图像中的源端语言文本翻译成目标语言。文本图像翻译系统通常由相互独立的光学字符识别(Optical Character Recognition，OCR)和机器翻译(Machine Translation，MT)模型级联组成。OCR模型将文本图像识别成转录文本，MT模型将转录文本翻译成目标语言。由于OCR模型转录文本存在噪声，而MT模型对噪声文本表现不佳，文本图像翻译系统性能远不如纯文本机器翻译系统。为缓解噪声文本带来的问题，鲁棒性机器翻译主要采用以下两种方法:①使用合成噪声文本，以模拟OCR转录带来的噪声;②利用干净文本和噪声文本的对比学习，拉近噪声文本和干净文本的分布。未能考虑以下问题:①忽视来自OCR模型的置信度信息，未能考虑OCR和MT系统的有效融合;②仅采用合成噪声，类型单一，无法覆盖实际噪声类型;③仅采用句子粒度的粗粒度对比损失，忽略细粒度的词的对比信息。为解决上述问题，该文提出一种融合置信度信息的文本图像翻译方法，充分利用转录文本中每个字符输出的概率分布，得到每个词的置信度信息，使级联式文本图像翻译系统中的OCR模型和机器翻译模型产生更有效的融合;同时针对OCR转录文本的噪声特点，设计了一种能提供词粒度的对比信息的监督文本，进一步提升模型性能。实验表明，该文所提方法在中译英以及英中文本图像翻译任务上相较于传统的管道式模型得到了显著提升。
Incorporating Confidence into Text Image Translation
Text image machine translation aims to translate the source language embedded in images into the target language.The text image translation system is usually a pipeline system by optical character recognition(OCR)and machine translation(MT)models.The OCR model recognizes the text image into a transcribed text,and then the MT model translates the transcribed text into the target language.As there are errors in the transcribed text from the OCR model and the NMT model is vulnerable to the errors in source transcribed texts from the OCR model.To address this issue,we propose a method to incorporate confidence information into text image translation which bridges the gap by reusing the ignored probability distribution of character in OCR to generate confidence for each token.Furthermore,in view of the characteristics of OCR recognition errors,we tailor a supervised text to provide contrastive information with word granularity to improve the systems'performance.Experimental results on Chi-nese-English and English-Chinese translation tasks demonstrate that our approach achieves significant improvements over the conventional pipeline methods.
NETL
NSTL
万方数据
