Neural Radiance Fields (NeRF) methods have proved effective as compact, high-quality and versatile representations for 3D scenes, and enable downstream tasks such as editing, retrieval, navigation, etc. Various neural architectures are vying for the core structure of NeRF, including the plain Multi-Layer Perceptron (MLP), sparse tensors, low-rank tensors, hashtables and their compositions. Each of these representations has its particular set of trade-offs. For example, the hashtable-based representations admit faster training and rendering but their lack of clear geometric meaning hampers downstream tasks like spatial-relation-aware editing. In this paper, we propose Progressive Volume Distillation (PVD), a systematic distillation method that allows any-to-any conversions between different architectures, including MLP, sparse or low-rank tensors, hashtables and their compositions. PVD consequently empowers downstream applications to optimally adapt the neural representations for the task at hand in a post hoc fashion. The conversions are fast, as distillation is progressively performed on different levels of volume representations, from shallower to deeper. We also employ special treatment of density to deal with its specific numerical instability problem. Empirical evidence is presented to validate our method on the NeRF-Synthetic, LLFF and TanksAndTemples datasets. For example, with PVD, an MLP-based NeRF model can be distilled from a hashtable-based Instant-NGP model at a 10X~20X faster speed than being trained the original NeRF from scratch, while achieving a superior level of synthesis quality. Code is available at https://github.com/megvii-research/AAAI2023-PVD.
翻译:各种神经结构正在争夺 NERF 的核心结构, 包括平坦的多层透视( MLP)、 稀疏的强压、 低级强压、 散发式及其构成。 这些表达方式都有其特定的权衡组合。 例如, 散列式20- 基面代表接受更快的培训和提供,但缺乏清晰的几何含义会阻碍下游任务,例如编辑、检索、导航等。 在本文件中,我们提议递进量蒸馏(PVD),这是一种系统化的蒸馏方法,允许不同结构之间的任何到任何转换,包括 MLP、 稀疏或低级强压、 散压及其构成。 因此, PVD 能够让下游应用基于原始的模型, 以临时方式优化本部任务中的神经演示。 转换是快速的, 由于蒸馏- 蒸馏- 亚氏度编辑( 速调色调), 我们的S- 快速化( ) 和 高压( 亚氏) 的内压( ) 直径) 的内压( RF), 直压( 直径) 直径) 直压(我们从一个特殊的解) 直径) 的内压(从10- 直压(RF) 直压(ro) 直压(RF) 直压(ro) 解)。