An aspirational goal for virtual reality (VR) is to bring in a rich diversity of real world objects losslessly. Existing VR applications often convert objects into explicit 3D models with meshes or point clouds, which allow fast interactive rendering but also severely limit its quality and the types of supported objects, fundamentally upper-bounding the "realism" of VR. Inspired by the classic "billboards" technique in gaming, we develop Deep Billboards that model 3D objects implicitly using neural networks, where only 2D image is rendered at a time based on the user's viewing direction. Our system, connecting a commercial VR headset with a server running neural rendering, allows real-time high-resolution simulation of detailed rigid objects, hairy objects, actuated dynamic objects and more in an interactive VR world, drastically narrowing the existing real-to-simulation (real2sim) gap. Additionally, we augment Deep Billboards with physical interaction capability, adapting classic billboards from screen-based games to immersive VR. At our pavilion, the visitors can use our off-the-shelf setup for quickly capturing their favorite objects, and within minutes, experience them in an immersive and interactive VR world with minimal loss of reality. Our project page: https://sites.google.com/view/deepbillboards/
翻译:虚拟现实( VR) 的渴望目标是将大量真实世界的物体无损地带入多样化。 现有的 VR 应用程序经常将对象转换成直立的 3D 模型, 使用 meshes 或点云, 允许快速互动翻转, 但也严重限制其质量和支持对象的类型, 从根本上将 VR 的“ 现实主义” 提升到“ 现实主义 ” 上方 。 在经典的“ 广告牌” 游戏游戏游戏中, 我们开发了“ 深广告板 ”, 模型3D 对象暗地使用神经网络网络, 并且根据用户的查看方向, 仅仅将 2D 图像转换成。 我们的系统, 将商用 VR 头像与正在运行神经转换的服务器连接起来, 允许对详细的僵硬对象、 发光线天体、 激活的动态物体进行实时高分辨率模拟, 并在互动 VR 世界上大幅缩小现有的“ 模拟” ( real2simal) 差距。 此外, 我们用物理互动游戏中的经典广告板板板板将屏幕调整为隐形 VR 。 在我们的游戏/ 中, 的游戏中可以快速地拍摄中快速地拍摄中, 。