VALSE 论文速览第45期：Neural Body：用带有隐式结构编码的INR生成动态人体新视角

2022 年 1 月 28 日 VALSE

为了使得视觉与学习领域相关从业者快速及时地了解领域的最新发展动态和前沿技术进展，VALSE最新推出了《论文速览》栏目，将在每周发布一至两篇顶会顶刊论文的录制视频，对单个前沿工作进行细致讲解。本期VALSE论文速览选取了来自浙江大学的3D视图合成方面的工作，该视频由论文第一作者浙江大学彭思达同学录制。

论文题目：Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans

作者列表：彭思达 (浙江大学)，张远青 (浙江大学)，徐英豪 (香港中文大学)，王倩倩 (康奈尔大学)，帅青 (浙江大学)，鲍虎军 (浙江大学)，周晓巍 (浙江大学)

B站观看网址：

https://www.bilibili.com/video/BV15F411p7H7/

复制链接到浏览器打开或点击阅读原文即可跳转至观看页面。

论文摘要：

This paper addresses the challenge of novel view synthesis for a human performer from a very sparse set of camera views. Some recent works have shown that learning implicit neural representations of 3D scenes achieves remarkable view synthesis quality given dense input views. However, the representation learning will be ill-posed if the views are highly sparse. To solve this ill-posed problem, our key idea is to integrate observations over video frames. To this end, we propose Neural Body, a new human body representation which assumes that the learned neural representations at different frames share the same set of latent codes anchored to a deformable mesh, so that the observations across frames can be naturally integrated. The deformable mesh also provides geometric guidance for the network to learn 3D representations more efficiently. To evaluate our approach, we create a multi-view dataset named ZJU-MoCap that captures performers with complex motions. Experiments on ZJU-MoCap show that our approach outperforms prior works by a large margin in terms of novel view synthesis quality. We also demonstrate the capability of our approach to reconstruct a moving person from a monocular video on the People-Snapshot dataset.

论文信息：

[1] Sida Peng, Yuanqing Zhang, Yinghao Xu, Qianqian Wang, Qing Shuai, Hujun Bao, Xiaowei Zhou, Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans. In CVPR, 2021.

论文链接：

[https://openaccess.thecvf.com/content/CVPR2021/papers/Peng_Neural_Body_Implicit_Neural_Representations_With_Structured_Latent_Codes_for_CVPR_2021_paper.pdf]

代码链接：

[https://github.com/zju3dv/neuralbody]

视频讲者简介：

彭思达是浙江大学CAD&CG国家重点实验室四年级博士研究生，导师为周晓巍研究员。研究方向为三维视觉，主要研究三维重建与视角合成。博士至今以一作身份在TPAMI、CVPR、ICCV等会议或期刊发表5篇论文，论文引用超过500次。在2020年获得CCF-CV学术新锐奖，全国仅评选3人。在2021年一作论文入围CVPR Best Paper Candidates。发表论文均已开源，在GitHub上Star数超过2000次。