自适应动态规划及在城市交通信号优化控制中的应用

项目名称： 自适应动态规划及在城市交通信号优化控制中的应用

项目编号： No.60874043

项目类型： 面上项目

立项/批准年度： 2009

项目学科： 电工技术

项目作者： 赵冬斌

作者单位： 中国科学院自动化研究所

项目金额： 30万元

中文摘要： 自适应动态规划(ADP)是利用神经网络来逼近动态规划的性能指标函数，实现最优控制的策略，有效地解决了动态规划维数灾的难题，为高维复杂系统的最优控制提供了一种切实可行的理论和方法，近年来其理论和方法的研究日益受到重视。另一方面，交通拥堵的问题日益突出，而关键问题之一在于现有的城市交通信号控制系统没有充分发挥合理的交通指挥和疏导作用，为自适应动态规划的有效应用提供了广泛空间。课题组以基金申请内容的基于自适应动态规划的城市交通信号优化控制为主线，展开了相关领域的系统研究，所获得的研究成果包括：基于计算智能方法的城市交通信号优化控制的国内外研究现状调研；监督式自适应动态规划方法；多个Agent强化学习方法；基于自适应动态规划的城市街区路口交通信号优化控制；基于自适应动态规划的城市快速路交通信号优化控制；所设计的交通信号控制方法在微观交通仿真平台上的实验验证等，取得了创新的成果。其中一些补充内容如基于监督式自适应动态规划的车辆自适应巡航控制，是结合城市交通信号优化控制的最新发展方向- - 车路一体化而进行的。课题研究成果体现在高水平的学术论文（包括IEEE汇刊4篇）、授权发明专利和软件著作权等。

中文关键词： 自适应动态规划;收敛性;分布式协调优化;交通信号控制

英文摘要： Adaptive dynamic programming (ADP) uses neural networks to approximate the performance index of dynamic programming (DP), aiming to achieve an optimal control strategy. Therefore, the notorious problem "curse of dimensionality" from DP is solved effectively by ADP, to provide a feasible theory and methodology for the optimal control of multi-dimensional complex systems. ADP receives more and more attention in researches and applications gradually. On the other hand, traffic is getting more congested. One of the key reasons lies in the malfunction of current urban traffic signal control systems, which invokes more efficient control strategies. This project aims at research issues on ADP based traffic signal control, and other related systematic aspects. The major achievements are as follows: the review and research direction on urban traffic signal control methodologies based on computational intelligence; supervised reinforcement learning method and applications; multi-agent reinforcement learning method; traffic signal control on urban streets based on ADP; freeway ramp metering based on ADP; the experimental verification on a micro-simulation platform with the proposed traffic signal controllers. Some new research contents are supplemented, e.g. supervised reinforcement learning on vehicle control, which is presented following the new research direction on V2R (vehicle to road) integration. The above achievements were embodied with patents, software registrations, papers in highly referred academic international journals (4 in IEEE), other journals and conferences, which made solid foundation for future research and implementation.

英文关键词： Adaptive Dynamic Programming; convegence; distributed coordinated optimization; traffic signal control