While deep reinforcement learning (RL) has been increasingly applied in designing car-following models in the last years, this study aims at investigating the feasibility of RL-based vehicle-following for complex vehicle dynamics and strong environmental disturbances. As a use case, we developed an inland waterways vessel-following model based on realistic vessel dynamics, which considers environmental influences, such as varying stream velocity and river profile. We extracted natural vessel behavior from anonymized AIS data to formulate a reward function that reflects a realistic driving style next to comfortable and safe navigation. Aiming at high generalization capabilities, we propose an RL training environment that uses stochastic processes to model leading trajectory and river dynamics. To validate the trained model, we defined different scenarios that have not been seen in training, including realistic vessel-following on the Middle Rhine. Our model demonstrated safe and comfortable driving in all scenarios, proving excellent generalization abilities. Furthermore, traffic oscillations could effectively be dampened by deploying the trained model on a sequence of following vessels.
翻译:虽然在过去几年里,在设计汽车跟踪模型时越来越多地应用了深度强化学习(RL),但本研究旨在调查基于RL的车辆跟踪复杂车辆动态和强烈环境扰动的可行性,作为使用实例,我们开发了一个基于现实的船舶动态的内陆水道船舶跟踪模型,该模型考虑到环境影响,例如不同的流速和河流状况等。我们从匿名的AIS数据中提取了自然船只行为,以形成一种奖励功能,反映舒适和安全航行旁边的现实驾驶风格。为了实现高度的通用能力,我们提议一个使用随机过程模拟引导轨道和河流动态的RL培训环境。为了验证经过培训的模型,我们界定了在培训中看不到的不同情景,包括在中莱茵号上现实的船舶跟踪。我们的模型在所有情景中都展示了安全舒适的驾驶方式,证明了极好的通用能力。此外,通过在跟踪船只的顺序上部署经过训练的模型,交通振荡可以有效地抑制交通振荡。