Traffic congestion is a serious problem in urban areas. Dynamic congestion pricing is one of the useful schemes to eliminate traffic congestion in strategic scale. However, in the reality, an optimal dynamic congestion pricing is very difficult or impossible to determine theoretically, because road networks are usually large and complicated, and behavior of road users is uncertain. To account for this challenge, this work proposes a dynamic congestion pricing method using deep reinforcement learning (DRL). It is designed to eliminate traffic congestion based on observable data in general large-scale road networks, by leveraging the data-driven nature of deep reinforcement learning. One of the novel elements of the proposed method is the distributed and cooperative learning scheme. Specifically, the DRL is implemented by a spatial-temporally distributed manner, and cooperation among DRL agents is established by novel techniques we call spatially shared reward and temporally switching learning. It enables fast and computationally efficient learning in large-scale networks. The numerical experiments using Sioux Falls Network showed that the proposed method works well thanks to the novel learning scheme.
翻译:交通堵塞是城市地区的一个严重问题。动态交通堵塞定价是消除战略规模交通堵塞的有用办法之一。然而,在现实中,最佳的动态交通堵塞定价很难或不可能在理论上确定,因为公路网络通常规模大而复杂,道路使用者的行为也不确定。考虑到这一挑战,这项工作提议采用动态交通堵塞定价方法,采用深层强化学习(DRL),目的是通过利用由数据驱动的深层强化学习在一般大型公路网络的可观测数据基础上消除交通堵塞。拟议方法的新内容之一是分配和合作学习计划。具体地说,DRL采用空间即时分配的方式实施,DRL代理机构之间的合作是通过我们称之为空间共享奖励和时间转换学习的新技术确立的。这有利于在大型网络中快速和计算高效学习。使用Sioux Falls网络进行的数字实验表明,拟议的方法在新学习计划下运作良好。