由于网络环境的多变性,视频播放过程中容易出现卡顿、比特率波动等情况,严重影响了终端用户的体验质量.为优化网络资源分配并提升用户观看体验,准确评估视频质量至关重要.现有的视频质量评价方法主要针对短视频,普遍关注人眼视觉感知特性,较少考虑人类记忆特性对视觉信息的存储和表达能力,以及视觉感知和记忆特性之间的相互作用.而用户观看长视频的时候,其质量评价需要动态评价,除了考虑感知要素外,还要引入记忆要素.为了更好地衡量长视频的质量评价,本文引入深度网络模型,深入探讨了视频感知和记忆特性对用户观看体验的影响,并基于两者特性提出长视频的动态质量评价模型.首先,本文设计主观实验,探究在不同视频播放模式下,视觉感知特性和人类记忆特性对用户体验质量的影响,构建了基于用户感知和记忆的视频质量数据库(Video Quality Data-base with Perception And Memory,PAM-VQD);其次,基于PAM-VQD数据库,采用深度学习的方法,结合视觉注意力机制,提取视频的深层感知特征,以精准评估感知对用户体验质量的影响;最后,将前端网络输出的感知质量分数、播放状态以及自卡顿间隔作为三个特征输入长短期记忆网络,以建立视觉感知和记忆特性之间的时间依赖关系.实验结果表明,所提出的质量评估模型在不同视频播放模式下均能准确预测用户体验质量,且泛化性能良好.
Research of Video Dynamic Quality Evaluation Based on Human Perception and Memory
Due to the variability of the network environment,video playback is prone to lag and bit rate fluctuations,which seriously affects the quality of end-user experience.In order to optimize network resource allocation and enhance us-er viewing experience,it is crucial to accurately evaluate video quality.Existing video quality evaluation methods mainly fo-cus on the visual perception characteristics of short videos,with less consideration of the ability of human memory charac-teristics to store and express visual information,and the interaction between visual perception and memory characteristics.In contrast,when users watch long videos,video quality evaluation needs dynamic evaluation,which needs to consider both perceptual and memory elements.To better measure the quality evaluation of long videos,we introduce a deep network model to deeply explore the impact of video perception and memory characteristics on users'viewing experience,and pro-poses a dynamic quality evaluation model for long videos based on these two characteristics.Firstly,we design subjective experiments to investigate the influence of visual perceptual features and human memory features on user experience quali-ty under different video playback modes,and constructs a video quality database with perception and memory(PAM-VQD)based on user perception and memory.Secondly,based on the PAM-VQD database,a deep learning methodology is utilized to extract deep perceptual features of videos,combined with visual attention mechanism,in order to accurately evaluate the impact of perception on user experience quality.Finally,the three features of perceptual quality score,playback status and self-lag interval output from the front-end network are fed into the long short-term memory network to establish the tempo-ral dependency between visual perception and memory features.The experimental results show that the proposed quality as-sessment model can accurately predict the user experience quality under different video playback modes with good general-ization performance.
visual perceptual propertiesmemory effectquality of experience(QoE)deep learningattention mech-anism
万方数据
维普
