Although autonomous vehicles (AVs) are expected to revolutionize transportation, robust perception across a wide range of driving contexts remains a significant challenge. Techniques to fuse sensor data from camera, radar, and lidar sensors have been proposed to improve AV perception. However, existing methods are insufficiently robust in difficult driving contexts (e.g., bad weather, low light, sensor obstruction) due to rigidity in their fusion implementations. These methods fall into two broad categories: (i) early fusion, which fails when sensor data is noisy or obscured, and (ii) late fusion, which cannot leverage features from multiple sensors and thus produces worse estimates. To address these limitations, we propose HydraFusion: a selective sensor fusion framework that learns to identify the current driving context and fuses the best combination of sensors to maximize robustness without compromising efficiency. HydraFusion is the first approach to propose dynamically adjusting between early fusion, late fusion, and combinations in-between, thus varying both how and when fusion is applied. We show that, on average, HydraFusion outperforms early and late fusion approaches by 13.66% and 14.54%, respectively, without increasing computational complexity or energy consumption on the industry-standard Nvidia Drive PX2 AV hardware platform. We also propose and evaluate both static and deep-learning-based context identification strategies. Our open-source code and model implementation are available at https://github.com/AICPS/hydrafusion.
翻译:虽然自治车辆(AV)有望使运输发生革命,但各种驾驶环境的强力认知仍是一个重大挑战。为了改善AV感知,已提议采用将摄影机、雷达和利达尔传感器的传感器数据装配成引信的技术来改进AV感知;然而,在困难的驾驶环境(例如恶劣天气、低光、感官障碍),现有方法不够健全,原因是其融合实施过程僵化。这些方法分为两大类:(一) 早期融合,当传感器数据吵闹或模糊时,这种融合失败,以及(二) 延迟融合,无法利用多个传感器的功能,从而产生更糟糕的估计数。为了应对这些限制,我们提议“水力动力聚合”:一个选择性传感器集成框架,学会确定目前的驱动环境,并结合最佳的传感器组合,以便在不损害效率的情况下实现最大程度的稳健性。“水力融合”是第一个提议在早期融合、迟化、静态融合和模型之间进行动态调整的办法,从而在应用深度和时间上都不同。我们指出,平均而言,DFusion(HFus)超出多个传感器的超出和晚版版本,我们标准化的A66%和硬化的消费平台将分别提议采用。我们的标准化的计算方法。