面向视频预测图像重建的感知损失函数
Perceptual loss for image reconstruction in video prediction
涂思仪 1黄劲松1
作者信息
- 1. 武汉大学 测绘学院,武汉 430079
- 折叠
摘要
为了进一步提升视频预测网络重建场景图像的精确度,提出一种利用自编码器的多尺度判别器特征感知损失函数(MDF-AE):其核心是用一组面向不同尺度图像的判别器构成提取图像特征的损失网络;判别器网络通过单图像生成对抗网络(SinGAN)分阶段训练得到,训练中以图像自编码器作为生成器,在生成图像中引入图像重建误差,为视频预测网络进行未来图像帧的重建提供更准确的感知约束.实验结果表明,利用 MDF-AE训练视频预测网络可有助于从结构、纹理和色彩上提升网络所重建场景图像的质量和可视化效果.
Abstract
In order to further improve the accuracy of scene image reconstruction by video prediction network,the paper proposed a multi-scale discriminator feature perceptual loss function with auto-encoder(MDF-AE):the core is to use a set of discriminators for images of different scales to form a loss network and extract image features;the discriminators were trained in stages by a single image generative adversarial network(SinGAN),and the image auto-encoder was used as the generator,then the image reconstruction error was introduced into the generated image for providing more accurate perceptual constraints on the image reconstruction when the network predicts future frames.Experimental results showed that using MDF-AE to train a video prediction network could help improve the quality and visualization effect of scene images reconstructed by the network in terms of structure,texture and color.
关键词
视频预测/感知损失/图像自编码器/单图像生成对抗网络/生成对抗训练Key words
video prediction/perceptual loss/image auto-encoder/single image generative adversarial network/generative adversarial training引用本文复制引用
出版年
2024
NETL
NSTL
万方数据