Progress of image retargeting quality evaluation:a survey
With the popularization of the Internet,intelligent technology,and various sensing devices,images and videos are playing an increasingly important role in video surveillance,health care,and distance education.All kinds of informa-tion can be obtained through images or videos using different terminal devices(such as mobile phones,laptops,and tab-lets)anytime and anywhere.Because different terminal devices have dissimilarsizes and resolutions,how to display the same image with high visual quality on these devices has become a common concern of the academic and industrial commu-nity.To solve this problem,image retargeting technology has emerged and become a popular,cutting-edge research direc-tion in the field of computer vision and image processing.Image retargeting technology aims to adjust the image resolution without destroying the visual content to adapt to the information acquisition of various terminals.Traditional image retarget-ing algorithms achieve this goal through some simple operations(such as scaling and clipping),but such operations usually cause serious distortions of visual content and the loss of important information;thus,obtaining a visual satisfying retar-geted image is difficult.To compensate for the performance disadvantages of traditional algorithms,a series of more advanced content-aware image retargeting algorithms have been proposed in recent years.Generally,these algorithms adopt a two-stage framework.The first step is to calculate the importance map of the input image and assign an importance weight to each pixel.The higher the weight is,the higher the probability that it should be retained.The second step is to implement the corresponding image resizing method to preserve the important content of the image as much as possible to meet the geometric constraints.According to different technologies,content-based image retargeting algorithms can be divided into four categories:discrete methods,continuous methods,multioperator methods,and deep learning methods.Although great progress has been made in this field,no algorithm can be guaranteed to meet the requirements of multiple display devices without reducing the visual quality.Distortions are inevitably introduced in image retargeting.Therefore,how to evaluate the quality of retargeted images objectively and accurately is very important for the selection,optimization,and development of image retargeting algorithms.Image quality assessment(IQA)is a basic problem in image processing and computer vision.In general,it can be divided into subjective assessment and objective assessment.Subjective assess-ment,which is the most direct,effective way,is completed through the judgment of subjects.Objective assessment is to predict and evaluate image quality automatically by constructing models.Compared with subjective assessment,objective assessment has the advantages of low cost,reusability,and easy deployment,so it is the focus of the existing research.According to whether reference image is used or not,it can be further divided into full-reference quality metrics,reduced-reference quality metrics,and no-reference quality metrics.According to different problems,it can also be divided into objective assessment of natural images,objective assessment of screen content images,objective assessment of cartoon images,and objective assessment of image retargeting.In recent years,a massive effort has been made in objective image retargeting quality assessment(IRQA);thus,encouraging research progress has been achieved.However,up to now,no review paper has studied image retargeting quality evaluation,which could hinder further development of this field.To this end,this review paper analyzes the challenges facing the field,reviews and summarizes the existing methods,examines their advantages and disadvantages,and states the possible development directions of the field to help researchers quickly understand and master the basic situation and promote the development of this field.First,the related work,namely,image retargeting and traditional IQA,is briefly introduced according to the classification principle.Specifically,image retargeting is divided into traditional algorithms and content-aware algorithms,and traditional IQA is described and summa-rized according to full-reference metric,reduced-reference metric,and no-reference metric.Second,the dataset and objec-tive evaluation of image retargeting are introduced emphatically.For the datasets,subjective quality assessment is used as the performance upper bound of objective quality assessment and provides performance comparison for objective quality algorithms.From these datasets,the distortion characteristics of image retargeting are analyzed,and the important steps in the construction of different databases are compared and summarized.Two types of representative IRQA models,namely,the traditional feature similarity-based IRQA model and the image registration-based IRQA model,are introduced and sum-marized in detail.Traditional feature similarity-based IRQA models extract perception-sensitive features from reference images and retargeted images,respectively and quantify the distortion degree of retargeted images by calculating their fea-ture similarity.To cater to the characteristics of the human visual system,saliency maps are often incorporated into the design of these models.Before feature extraction,image registration-based IRQA models first conduct sparse or dense image registration of two images to extract features effectively and improve the outcome of the algorithms.Next,their main ideas,advantages,and disadvantages are thoroughly analyzed and compared.Third,the performances of representative IRQA metrics are compared and analyzed on three public datasets in terms of prediction accuracy and monotonicity.Experi-mental results show the current models are only partially consistent with human perception and can still be improved.More energy and efforts are needed in this field.Finally,the current problems and challenges in the field of 1RQA are summa-rized,and the possible development directions in the future are identified.
NETL
NSTL
万方数据
