文本识别技术可以分为光学字符识别(optical character recognition，OCR)和场景文本识别(scene text recog-nition，STR)，其中STR是在OCR基础上针对日益复杂的应用场景衍生出来的。依托深度学习，OCR技术近年来取得了长足进步并大规模商业落地，但深度学习面临的对抗样本攻击问题也给OCR带来了安全威胁。目前大多数OCR模型均存在识别自然扰动和防御对抗样本攻击能力差的问题，如OCR模型在噪声、水印和梯度等攻击算法下的识别准确率大大降低。相比图像领域，文本识别领域的对抗样本攻击研究还远远不够。文本识别通常被视为一个序列到序列的问题，其中输入(如图像中的像素)和输出(像素对应的字符)都是序列，这使得对抗样本的生成更具挑战性。本文对文本识别的对抗样本攻击和防御方法进行研究综述，梳理了近年来文本识别领域的对抗样本攻击方法并进行对比分析，根据攻击类型、应用场景和模型可知性，对攻击方式进行了系统分类。具体来说，按照攻击类型，可分为基于梯度的攻击、基于优化的攻击和基于生成模型的攻击;按照应用场景，可以分为OCR攻击和STR攻击;按照模型可知性，可分为白盒攻击和黑盒攻击。除了回顾文本识别对抗样本攻击方法，还简要介绍了防御技术，具体分为数据预处理、文本篡改检测和传统对抗防御技术。通过这些技术的应用，可以有效地提升文本识别模型的安全性和鲁棒性。最后，总结了文本识别领域对抗样本攻击及防御面临的挑战，并对未来发展方向做出展望。
A review of adversarial examples for optical character recognition
In the context of deep learning,an increasing number of fields are adopting deep and recurrent neural networks to construct high-performance data-driven models.Text recognition is widely applied in daily life fields,such as autono-mous driving,product retrieval,text translation,document recognition,and logistics sorting.The detection and recogni-tion of text from scene images can considerably reduce labor costs,improve work efficiency,and promote the development of information intelligence.Therefore,the research on text detection and recognition technology has practical and scientific value.The field of text recognition has resulted in the use of methods from recognition and sequence networks,which led to evolving technologies,such as methods based on connectionist temporal classification(CTC)loss,those based on attention mechanisms,and end-to-end recognition.CTC-and attention-based approaches perceive the task of matching text images with the corresponding character sequences as a sequence-to-sequence recognition issue by employing an encoder for the process.End-to-end text recognition methods meld text detection and recognition modules into a unified model,which facilitates the simultaneous training of both modules.Although the advancement of deep learning has driven the develop-ment of optical character recognition(OCR)technology,some researchers have discovered serious vulnerabilities in deep models:The addition of minute disturbances to images can cause the model to make incorrect judgments.In applications demanding a high performance,this phenomenon greatly hinders the application process of deep models.Therefore,an increasing number of researchers are beginning to focus on strategies for understanding the deep model's response to this anomaly.Before understanding how the model resists this disturbance's performance,a key task is discovering a mecha-nism for better disturbance generate,i.e.,how to attack the model.Thus,most current research focuses on the develop-ment of algorithms that can generate disturbances efficiently.This article reviews and summarizes various adversarial examples of attack methods proposed in the field of text recognition.Approaches to adversarial attacks are divided into three types:gradient-,optimization-,and generative model-based types.These categories are further delineated into white-and black-box techniques,which are contingent upon the level of access to model parameters.In the field of text rec-ognition,prevalent attack strategies involve watermark tactics and cleverly embed disturbances within the watermark.This approach maintains the attack success rate whilst rendering the adversarial image perceptibly natural to the human observer.Common attack methods also include additive and erosion disturbances and minimal pixel attack methods.Gen-erative adversarial network-based attacks have contributed to the research on English and Chinese font-style generation.They deceive machine learning models by producing examples similar to the original data and thereby improve the robust-ness and reliability of OCR models.The research on Chinese font-style conversion can be attributed to three categories:1)Stroke trajectory-based Chinese character generation methods generate novel characters by scrutinizing the stroke trajectory inherent to Chinese script.These techniques harness the unique stroke traits of Chinese characters to engender properties with similar stylistic attributes to accomplish style transference;2)Style-based Chinese character generation methods gen-erate new Chinese characters of specific style by learning the style features of various fonts;3)Methods based on content and style features generate Chinese characters with specific style and content by learning the representation of content and style features.The attack of OCR adversarial examples provoked reflections on the security of neural networks.Some defense methods include data preprocessing,text tampering detection,and traditional adversarial sample defense meth-ods.Finally,this review summarizes the challenges faced by adversarial sample attacks and defenses in text recognition.In the future,the transition from a white-box environment to a black-box environment requires extreme amount of attrac-tion.In classification models,the content of black-box queries is relatively direct object,with only the unnormalized logi-cal output of the last layer of the model needed to be obtained.However,sequence tasks are incapable of performing single-step output,which makes more effective attacks in fewer query environments a challenging problem.Considerable advance-ments have been attained in response generation during the advent of substantial vision-language models,such GPT-4.Regardless,the associated privacy and security concerns warrant attention,and thus,the adversarial robustness of large models needs further research.This review aims to provide a comprehensive perspective for the comprehension and resolu-tion of adversarial problems in recognition to find the right balance between practicality and security and promote the con-tinuous progress of the field.
optical character recognition(OCR)scene text recognition(STR)adversarial examplesgenerative adver-sarial network(GAN)deep learningsequence model
NETL
NSTL
万方数据
