End-to-end approaches to autonomous driving commonly rely on expert demonstrations. Although humans are good drivers, they are not good coaches for end-to-end algorithms that demand dense on-policy supervision. On the contrary, automated experts that leverage privileged information can efficiently generate large scale on-policy and off-policy demonstrations. However, existing automated experts for urban driving make heavy use of hand-crafted rules and perform suboptimally even on driving simulators, where ground-truth information is available. To address these issues, we train a reinforcement learning expert that maps bird's-eye view images to continuous low-level actions. While setting a new performance upper-bound on CARLA, our expert is also a better coach that provides informative supervision signals for imitation learning agents to learn from. Supervised by our reinforcement learning coach, a baseline end-to-end agent with monocular camera-input achieves expert-level performance. Our end-to-end agent achieves a 78% success rate while generalizing to a new town and new weather on the NoCrash-dense benchmark and state-of-the-art performance on the more challenging CARLA LeaderBoard.
翻译:自主驾驶的端对端方法通常依赖专家演示。虽然人类是良好的驱动力,但他们不是需要密集政策监督的端对端算法的好导师。相反,利用特权信息的自动化专家可以有效地产生大规模政策和非政策演示。然而,现有的城市驾驶自动化专家大量使用手工制作的规则,甚至对有地面真实信息的驾驶模拟器也进行副最佳操作。为解决这些问题,我们培训了一名强化学习专家,该专家将鸟类的视景图像绘制成持续低水平行动图。在为CARLA设定新的高级性能的同时,我们的专家还是一名更好的导师,为模仿学习者学习提供信息性能的监督信号。在我们的强化学习教练的监督下,一个具有单层摄像作用的基线端对端代理实现了专家级业绩。我们的端对端代理获得了78 %的成功率,同时将诺克拉希敏度基准和更具有挑战性的CARA领导人的状态表现概括为一个新的城镇和新的天气。