基于深度生成模型的视觉模式表示与编码
Visual Pattern Representation and Coding Based on Deep Generative Models
郭怡琳 1常建慧 2黄成 3马思伟4
作者信息
- 1. 北京大学深圳研究生院,中国 深圳 518055
- 2. 北京大学,中国 北京 100871
- 3. 中兴通讯股份有限公司,中国 深圳 518057
- 4. 北京大学,中国 北京 100871;鹏城实验室,中国 深圳 518057
- 折叠
摘要
认为早期智能编码方法的性能受限于手工设计的方案,当前基于神经网络的编码方法可解释性不足,不利于后续面向人机视觉的分析与交互.受生成模型的启发,生成式编码方法通过构建生成模型来实现图像和视频的压缩和合成,获得可解释的紧凑视觉表示并生成符合图像先验分布的高视觉质量内容.其中概念图像编码与概念视频编码利用生成模型强大的样本生成能力与紧凑层次视觉表示模型,实现了编码性能更优的图像与视频编码;跨模态语义编码对图像与文本域进行跨模态转换与编码,保持可解释的同时实现上千倍的超高压缩比与令人满意的重构结果.
Abstract
The performance of early intelligent encoding methods was limited by manually designed solutions,while current neural network-based encoding methods lack interpretability,which hinders subsequent analysis and interaction between humans and machine vision.In-spired by generative models,the generative encoding methods aim to achieve compression and synthesis of images and videos by con-structing efficient generative models,obtaining interpretable compact visual representations,and synthesizing high-quality visual content that conforms to the prior distribution of images.Among them,conceptual image encoding and conceptual video encoding leverage the pow-erful sample generation capability and compact hierarchical visual representation models of generative models,resulting in superior encod-ing performance for images and videos.Cross-modal semantic coding,on the other hand,enables cross-modal transformation and coding between the image and text domains while maintaining interpretability,achieving ultra-high compression ratios of thousands of times and satisfactory reconstruction results.
关键词
智能视频编码/生成式编码/跨模态压缩/概念编码Key words
intelligent video encoding/generative encoding/cross-modal compression/conceptual coding引用本文复制引用
基金项目
国家自然科学基金项目(62025101)
鹏城实验室重大攻关项目(PCL2024A02)
出版年
2024
NETL
NSTL
万方数据