基于级联优化网络的视频合成方法
Video Synthesis Method Based on Cascade Refinement Network
郝炯辉 1王国刚 1汪滢 1赵怀慈2
作者信息
- 1. 沈阳化工大学信息工程学院,辽宁沈阳 110142
- 2. 中国科学院沈阳自动化研究所,辽宁沈阳 110016
- 折叠
摘要
针对视频到视频的生成过程中视频生成质量较差,生成的物体属性无法在后续视频中得以延续,使仿真视频的视觉效果下降的问题,在图像到图像合成算法的基础上提出一种高分辨率的视频到视频的生成方法.在级联优化网络中增加残差块优化网络结构,从而提高生成视频帧的质量.为解决后续视频中生成物体属性不一致的问题,由两帧改进的级联优化网络预测图像计算光流,再由光流预测一帧图像,将这两个预测图像融合,得到仿真视频序列.与其他视频及图像生成方法在Cityscapes数据集上进行实验对比,结果表明所提算法可以得到更加真实的视频,并且生成的视频序列评价更高.
Abstract
A high-resolution video to video generation method is proposed based on the image to im-age synthesis algorithm to address the problem of poor video generation quality and inability to continue the generated object attributes in subsequent videos,resulting in a decrease in the visual effect of simulated videos.Adding residual blocks to the cascaded optimization network to optimize the network structure and improve the quality of generated video frames.In order to solve the problem that the attributes of the gen-erated objects are inconsistent in subsequent videos,the optical flow is calculated by two improved casca-ded optimization network prediction images,and then one image is predicted by optical flow.The two pre-dicted images are fused to obtain the simulation video sequence.Compared with other video and image synthesis methods on cityscapes dataset,the results show that the proposed algorithm can get more realistic video,and the generated video sequences have higher evaluation.
关键词
深度学习/视频合成/风格转换/光流估算Key words
deep learning/video to video synthesis/image style transfer/optical flow estimation引用本文复制引用
出版年
2024
NETL
NSTL
万方数据