Literature review of contextual information construction and applications in mixed reality
With the development of information technology,mixed reality(MR)technology has been applied in various fields,such as healthcare,education,and assisted guidance.MR scenes contain rich semantic information,and MR tech-nology based on scene context information can improve users'perception of the scene,optimize user interaction,and enhance the accuracy of interaction models.Therefore,they have quickly gained widespread attention.However,litera-ture reviews specifically investigating context information in this field are limited,and organization and classification are lacking.This paper focuses on MR technology and systems that utilize context information.This study was conducted fol-lowing the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.First,keywords for the search were determined on the basis of three factors:research domain,study subjects,and research scenarios.Subsequently,searches were performed in two influential databases in the field of MR:ACM Digital Library and IEEE Xplore.A prelimi-nary screening was then executed,considering the types of journals and conferences to eliminate irrelevant and unpub-lished literature.Subsequently,the titles and abstracts of the articles were reviewed sequentially,eliminating duplicates and irrelevant results.Finally,a total of 210 articles were individually screened to select 29 papers for the review.Addi-tionally,four more articles were included on the basis of expertise,resulting in a total of 33 articles for the review.Through a comprehensive literature review of MR databases,three research questions were formulated,and a dataset of research articles was established.The three research questions addressed in this paper are as follows:1)What are the different types of scene context?2)How is scene context organized in various MR technologies and systems?3)What are the application areas of empirical research?On the basis of the evolution of scene context and the refinement of MR technolo-gies and systems,we analyze the empirical research papers spanning nearly 20 years.This analysis involves summarizing previous research and providing an overview of the latest developments in systems that leverage scene context.We also pro-pose potential classification criteria,such as types of scene context,construction methods of knowledge bases for contex-tual information,fundamental technologies,and application domains.Among the various types of scene context,we cat-egorize them into six classes:scene semantics,object semantics,spatial relationships,group relationships,dependence relationships,and motion relationships.Scene semantics is the semantic information encompassed by various elements in the scene environment,including objects,characters,and texture information.In the categorization of object semantics,we consider information about the individual object itself,such as user information,type,attributes,and special content.Spatial relationship refers to numerical information,such as the relative position,angle,or arrangement between various objects in the scene.We analyzed spatial relationships in three ways:base spatial relationships,microscene spatial infor-mation,and real-scene spatial information.We consider a certain number of closely neighboring objects of the same cat-egory as a group.Group relations focus on information about the overall perspective such as intergroup relations and the number of groups.Dependence relationship is concerned with the dependencies and affiliations that may exist between dif-ferent objects in the scene at the functional and physical levels.Motion information is a new type of scene context,includ-ing basic motion information and special motion information,which describes the dynamic information of scene objects.Through an analysis of the utilization of various types of scene context,we establish the relationship between research objec-tives and contextual information,providing guidance on the selection of contextual information.The construction of knowl-edge bases is examined from user-intervention perspectives and types of fundamental technologies.Knowledge bases estab-lished with user intervention typically rely on researchers'abstract analysis of scene objects rather than pre-existing data-bases.Conversely,knowledge bases built without user intervention rely on existing information,such as low-level raw data in databases or predefined scenarios.The underlying technologies in this context are categorized into virtual reality(VR)and augmented reality(AR).Conducting classification research from the dual perspectives of user intervention and funda-mental technology facilitates a deeper understanding of how contextual information is organized in various MR systems.Application areas are investigated on the basis of the types of scenarios and whether they involve generative processes or not.The types of application scenarios are then categorized into six types:auxiliary guidance,AR annotation,scene recon-struction,medical treatment,object manipulation,and general purpose.Generative models can automatically generate tar-get information,such as AR-annotated shadows based on the scene,whereas nongenerative models mainly focus on specific operations.Through analysis from these two perspectives,the advantages and disadvantages of MR systems and technolo-gies in different application scenarios can be explored.Drawing upon the exploration and research in these three dimen-sions,we investigate the challenges associated with selecting,acquiring,and applying contextual information in MR sce-narios.By classifying the research objects from different dimensions,we address the research questions and identify cur-rent shortcomings and future research directions.The aim of this review is to support researchers across diverse fields in designing,selecting,and evaluating scene context,ultimately fostering the advancement of future MR application technolo-gies and systems.
virtual reality(VR)augmented reality(AR)perception and interactioncontext informationscene seman-tics
万方数据
维普
