We present MovingParts, a NeRF-based method for dynamic scene reconstruction and part discovery. We consider motion as an important cue for identifying parts, that all particles on the same part share the common motion pattern. From the perspective of fluid simulation, existing deformation-based methods for dynamic NeRF can be seen as parameterizing the scene motion under the Eulerian view, i.e., focusing on specific locations in space through which the fluid flows as time passes. However, it is intractable to extract the motion of constituting objects or parts using the Eulerian view representation. In this work, we introduce the dual Lagrangian view and enforce representations under the Eulerian/Lagrangian views to be cycle-consistent. Under the Lagrangian view, we parameterize the scene motion by tracking the trajectory of particles on objects. The Lagrangian view makes it convenient to discover parts by factorizing the scene motion as a composition of part-level rigid motions. Experimentally, our method can achieve fast and high-quality dynamic scene reconstruction from even a single moving camera, and the induced part-based representation allows direct applications of part tracking, animation, 3D scene editing, etc.
翻译:我们展示了移动部分,这是动态场景重建和部分发现的一种基于 NERF 的动态场景重建方法。 我们认为运动是确定部件的重要提示, 同一部分的所有粒子都具有共同运动模式。 从流体模拟的角度来看, 现有的动态 NERF 的变形方法可以被视为Eularian 视图下的场景运动参数, 即侧重于流体流经过的空间特定位置, 然而, 利用 Eulelirian 显示器提取构成物体或部件的动作是难以操作的。 在这项工作中, 我们引入了双轨视图, 在 Eulerian/ Lagrangian 视图下强制进行表达。 在 Lagrangian 视图下, 我们通过跟踪物体颗粒的轨迹, 将场景运动参数化为参数。 Lagrangian 视图使得通过将场景运动作为部分硬动作的构成要素来发现部分部分。 实验性地, 我们的方法可以实现快速和高质量的动态场景重建, 甚至从一个移动的相机中实现, 并且导出部分代表器可以直接应用部分跟踪、 动动画、 3D 场景编辑等。</s>