自由视点三维视频中纹理-深度图像联合建模及应用

项目名称： 自由视点三维视频中纹理-深度图像联合建模及应用

项目编号： No.61503294

项目类型： 青年科学基金项目

立项/批准年度： 2016

项目学科： 自动化技术、计算机技术

项目作者： 杨勐

作者单位： 西安交通大学

项目金额： 22万元

中文摘要： 自由视点三维视频技术是三维视频的最新解决方案。该技术从若干个特定视点采集目标场景的纹理、深度图像，然后用二者合成其它观看视点的虚拟视频。目前，虚拟视频质量较低是制约该技术发展的瓶颈之一。传统方法一般通过分别处理纹理、深度图像来优化虚拟视频的质量。但在实际系统中，纹理、深度图像相互作用，并同时影响虚拟视频的质量。针对该问题，本项目首先从联合分析纹理、深度图像之间的时空域关系角度建模虚拟视频质量的一系列变化特性，包括：1）联合分析二者对虚拟视频的影响，建立其质量估计的解析模型；2）联合分析二者的空域纹理差异，建模虚拟视频质量的局部区域重要性；3）基于二者空域结构相似特点联合建模它们之间的时域冗余信息；4）基于多目标优化理论建立二者质量的联合优化模型。然后，把以上模型应用于自由视点三维视频系统中的视点合成和视频编码过程，以改善合成的虚拟视频的质量，推动该技术的发展和在三维视频监控等领域中的应用。

中文关键词： 自由视点三维视频；视点合成；三维视频监控；视频编码

英文摘要： Free-viewpoint 3D (FV-3D) video is considered the most promising solution for 3D video in the future. In FV-3D, the texture and depth images of multiple views are captured, then they are used to synthesize the virtual videos of other desired views. In recent years, the development of FV-3D technique is slowed down due to various reasons, among which the low quality of the synthesized virtual videos has been the key point. Existing solutions try to improve its quality by processing texture image and depth image separately, supposing their effects on the virtual video are independent. However, in practical application, they actually interact with each other, and also affect the virtual video quality simultaneously. In this project, we aim to improve the virtual video quality by jointly considering the effects of texture image and depth image. First, we establish an analytical quality model by jointly considering their effects on the virtual video, such that the virtual video quality can be estimated directly and accurately. Second, we establish a regional quality model by jointly considering their spatial characteristics, such that the effect of different regions on the virtual video quality can be well differentiated. Third, we establish a temporal redundancy model to describe the mutual redundancy between texture image and depth image based on the similarity of their spatial geometry. Fourth, we establish the joint optimization model of both texture and depth image qualities based on Pareto optimization theory, such that their qualities can be optimized simultaneously. These models are then implemented in the view synthesis process and video coding process in FV-3D video system to optimize the virtual video quality, and then promote the development of FV-3D video technique with its industrial application, such as 3D video surveillance, etc.

英文关键词： Free-viewpoint 3D video;View synthesis;3D video surveillance;Video codng