In this paper, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of the sensors, we designed a fully physics-grounded simulation pipeline that includes material acquisition, ray-tracing-based infrared (IR) image rendering, IR noise simulation, and depth estimation. The pipeline is able to generate depth maps with material-dependent error patterns similar to a real depth sensor in real time. We conduct real experiments to show that perception algorithms and reinforcement learning policies trained in our simulation platform could transfer well to the real-world test cases without any fine-tuning. Furthermore, due to the high degree of realism of this simulation, our depth sensor simulator can be used as a convenient testbed to evaluate the algorithm performance in the real world, which will largely reduce the human effort in developing robotic algorithms. The entire pipeline has been integrated into the SAPIEN simulator and is open-sourced to promote the research of vision and robotics communities.
翻译:在本文中,我们侧重于模拟活跃立体深度传感器,这种传感器在学术界和工业界都十分流行。在传感器基本机制的启发下,我们设计了一个完全物理的模拟管道,其中包括材料获取、以射线追踪为基础的红外图像成像、IR噪声模拟和深度估计。管道能够产生与实时真正深度传感器类似的材料依赖错误模式的深度地图。我们进行了真正的实验,以表明在模拟平台上培训的感知算法和强化学习政策可以在不作任何微调的情况下顺利地向真实世界测试案例转移。此外,由于这种模拟的高度真实性,我们的深度传感器模拟器可以用作评估真实世界算法表现的方便测试台,这将大大降低人类开发机器人算法的努力。整个管道已经纳入SAPIEN模拟器,并且可以公开来源,以促进视觉和机器人社区的研究。