VisFly: An Efficient and Versatile Simulator for Training Vision-based Flight

We present VisFly, a quadrotor simulator designed to efficiently train vision-based flight policies using reinforcement learning algorithms. VisFly offers a user-friendly framework and interfaces, leveraging Habitat-Sim's rendering engines to achieve frame rates exceeding 10,000 frames per second for rendering motion and sensor data. The simulator incorporates differentiable physics and seamlessly integrates with the Gym environment, facilitating the straightforward implementation of various learning algorithms. It supports the direct import of all open-source scene datasets compatible with Habitat-Sim, enabling training on diverse real-world environments and ensuring fair comparisons of learned flight policies. We also propose a general policy architecture for three typical flight tasks relying on visual observations, which have been validated in our simulator using reinforcement learning. The simulator will be available at [https://github.com/SJTU-ViSYS/VisFly].