目的 新兴视频服务的功能参数设置将会直接影响到用户的认知状态，进一步影响用户体验质量，称为功能性用户体验质量(functional quality of experience，fQoE)。脑电信号蕴含丰富的大脑活动信息，能够揭示复杂脑活动中的脑网络模式，为fQoE提供可靠的评估依据。为此，本文首次提出了一个基于脑电的脑网络构建方法以评估fQoE，并研究fQoE背后的神经机制。方法 首先，通过改变功能参数诱发不同水平的fQoE，并同步收集脑电数据;然后，从脑电数据中提取单电极和多电极特征并以图结构进行融合，用以全面表征用户使用视频服务时的大脑状态;最后，使用基于自注意力图池化的脑网络构建模型来识别对fQoE敏感的脑网络，为fQoE提供可解释性，并进行分类以完成fQoE评估。结果 本文以弹幕视频服务的弹幕覆盖率这一功能参数为例验证了方法的科学性和可行性。实验表明，提出的评估方法在多种视频类型的fQoE评估中均达到了满意的效果，最佳识别准确率分别为86％(鬼畜类)、81％(科技类)、80％(舞蹈类)、82％(影视类)和84％(音乐类)。结论 来自fQoE相关的脑网络分析结果表明，额极、额中回、顶叶和颞叶的脑连接数量减少预示着观看弹幕视频的fQoE更高，即观看体验更好，同时也证明了功能参数通过影响人的脑状态进一步导致了fQoE的改变。本文的评估方法为fQoE的精确评估和视频服务功能参数的优化提供了来自神经生理学的定量工具和理论依据。
A brain network construction method for the assessment of functional quality of experience
Objective The rapid development of multimedia technology enables emerging video services,such as bullet chatting video and virtual idol.Technical parameters from network and application layers affect user's quality of experi-ence(QoE).In addition,the QoE changes when various adjustable functional parameters of these video services are modi-fied,which we call QoE influenced by functional parameters(functional QoE,fQoE).Changes in functional parameters influence human cognition and emotion,and thus,fQoE is almost entirely decided by human subjective perceptions.Infer-ring directly from parameter design,which is difficult,makes fQoE modeling challenging.The success of the video ser-vices depends entirely on user ratings,and understanding fQoE and the reason behind its generation is crucial for service providers.Studies using questionnaire and interview methods have been conducted to understand users'perceptions of functional parameters.However,subjects may be influenced by external criteria and social desirability,which potentially result in bias of the collected results.The above methods cannot perform the quantitative assessment of fQoE nor provide scientific evidence with interpretability.Electroencephalography(EEG)signals contain a wealth of information about brain activity,and EEG features can reveal brain network patterns during complex brain activity.Studies have used EEG as a powerful tool to assess the QoE influenced by technical parameters(tQoE)and uncovered relevant EEG single-electrode features,which revealed the correlation between tQoE and basic human perceptual functions and demonstrated the strong potential of EEG for fQoE assessment.However,fQoE may involve higher-order human cognitive functions that require interactions between multiple brain regions,such as social communication and emotion,whose complex relationships are difficult to be represented by single-electrode features.To address the limitations of the above studies,this paper presents an fQoE assessment model based on the EEG technology and investigates the neural mechanisms behind fQoE.Method First,an EEG dataset for fQoE assessment was constructed.To reduce the influence of subjects'personal preferences on the experimental results,we ensured that the stimulus materials contained five types of videos(auto-tune,technology,dance,film,and music).Different levels of fQoE were induced by changing the functional parameters,and the EEG data of subjects were collected simultaneously.Second,on top of single-electrode features,we additionally extracted multielec-trode features(i.e.,functional connectivity features)and fused both types in the form of graph.The EEG electrodes were represented as nodes on the graph,the single-electrode features as node features,and the functional connectivity as edges of the graph.The weights of edges represent the strength of functional connectivity,and were used in the comprehensive characterization of the user's brain state when using video service.Finally,self-attention graph pooling mechanism was introduced to construct a brain-network construction model to identify fQoE levels.The graph pooling layer can enlarge the field of view to the whole graph structure during the training process,retain key nodes,and compose new graphs to render the model with the capability to capture key brain networks.We further explored the neurophysiological principles behind it and provided theoretical support for the improvement of emerging video services.Result With the bullet chatting video,a new type of video service,as an example,this paper explored the fQoE affected by the functional parameter of bullet chat-ting coverage and the neurophysiological principles behind it.The finding verified the scientific validity and feasibility of the method.Experiments revealed that the assessment method proposed in this paper achieved satisfactory results in the fQoE evaluation of multiple video types,with the best recognition accuracy of 86％(auto-tune),80％(technology),80％(dance),82％(film),and 84％(music).Compared with existing machine learning and deep learning models,our method achieves the best recognition accuracy.The results on fQoE-related brain network analysis reveal that the number of brain connections in the frontal,parietal,and temporal lobes decreased,which indicates a attained fQoE for viewing bul-let chat videos,i.e.,a better viewing experience.This result also implies that functional parameters further lead to changes in the fQoE by affecting the human brain state.Specifically,the brain connection between the frontal and temporal lobes is related to speech information processing,the parietal lobe brain connection to visual information processing,the strength of frontal lobe brain connection to cognitive load levels,and its asymmetry to emotions and motivation.Conclusion In this study,we initially presented an EEG-based fQoE assessment model to evaluate the fQoE levels using a brain-network construction model based on a self-attention graph pooling mechanism and analyzed the neurophysiological ratio-nale behind it.The assessment method introduced in this paper serves as a quantitative tool and theoretical basis from neu-rophysiology for the accurate assessment of fQoE and optimization of functional parameters of video services.
emerging video servicesfunctional quality of experience(fQoE)electroencephalogram(EEG)brain net-work construction
NETL
NSTL
万方数据
