We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and physical simulation. Compared to other 3D representations like voxels and point clouds, meshes are more desirable in practice, because (1) they enable easy and arbitrary manipulation of shapes for relighting and simulation, and (2) they can fully leverage the power of modern graphics pipelines which are mostly optimized for meshes. Previous scalable methods for generating meshes typically rely on sub-optimal post-processing, and they tend to produce overly-smooth or noisy surfaces without fine-grained geometric details. To overcome these shortcomings, we take advantage of the graph structure of meshes and use a simple yet very effective generative modeling method to generate 3D meshes. Specifically, we represent meshes with deformable tetrahedral grids, and then train a diffusion model on this direct parametrization. We demonstrate the effectiveness of our model on multiple generative tasks.
翻译:我们考虑生成逼真的3D形状的任务,这对于自动场景生成和物理模拟等各种应用非常有用。与诸如体素和点云等其他3D表示相比,网格在实践中更可取,因为(1)它们可以轻松和任意地操纵形状以进行重新照明和模拟,(2)它们可以完全利用现代图形管道的强大功能,这些管道大多数是针对网格进行优化的。以前实现网格生成的可扩展方法通常依赖于次优的后处理,并且它们往往会产生过于平滑或嘈杂的表面,而没有精细的几何细节。为了克服这些缺点,我们利用网格的图结构,并使用一种简单但非常有效的生成建模方法来生成3D网格。具体而言,我们将网格表示为可变形四面体网格,然后在这种直接的参数化上训练扩散模型。我们在多个生成任务上展示了我们模型的有效性。