基于Dueling Double DQN的交通信号控制方法
Traffic Signal Control Method based on Dueling Double DQN
叶宝林 1陈栋 1刘春元 2陈滨 2吴维敏3
作者信息
- 1. 浙江理工大学信息科学与工程学院,杭州 310018;嘉兴大学信息科学与工程学院嘉兴市智慧交通重点实验室,浙江嘉兴 314001
- 2. 嘉兴大学信息科学与工程学院嘉兴市智慧交通重点实验室,浙江嘉兴 314001
- 3. 浙江大学智能系统与控制研究所,杭州 310027
- 折叠
摘要
为了提高交叉口通行效率缓解交通拥堵,深入挖掘交通状态信息中所包含的深层次隐含特征信息,提出了一种基于Dueling Double DQN(D3QN)的单交叉口交通信号控制方法;构建了一个基于深度强化学习Double DQN(DDQN)的交通信号控制模型,对动作-价值函数的估计值和目标值迭代运算过程进行了优化,克服基于深度强化学习DQN的交通信号控制模型存在收敛速度慢的问题;设计了一个新的Dueling Network解耦交通状态和相位动作的价值,增强Double DQN(DDQN)提取深层次特征信息的能力;基于微观仿真平台SUMO搭建了一个单交叉口模拟仿真框架和环境,开展仿真测试;仿真测试结果表明,与传统交通信号控制方法和基于深度强化学习DQN的交通信号控制方法相比,所提方法能够有效减少车辆平均等待时间、车辆平均排队长度和车辆平均停车次数,明显提升交叉口通行效率.
Abstract
In order to improve the efficiency of intersection traffic,alleviate traffic congestion,and deeply explore the deep hidden feature information contained in traffic status information,a single intersection traffic signal control method based on Dueling double DQN(D3QN)is proposed.A traffic signal control model based on deep reinforcement learning double DQN(DDQN)was construc-ted,and the iterative operation process of the target value and estimated value of the action value function was optimized to overcome the problem of slow convergence speed in the traffic signal control model based on deep reinforcement learning DQN.A new Dueling network was designed to decouple the value of traffic states and phase actions,enhancing the ability of the double DQN(DDQN)to extract the deep level feature information.On the basis of the micro simulation platform simulation of urban mobility(SUMO),sin-gle intersection simulation framework and environment were built to simulate the test.The simulation test results show that compared with traditional traffic signal control methods and traffic signal control methods based on the deep reinforcement learning DQN,the proposed method can effectively reduce the average waiting time,average queue length,and mean stops of vehicles,significantly im-proving the efficiency of intersection traffic.
关键词
交通信号控制/深度强化学习/Dueling/Double/DQN/Dueling/NetworkKey words
traffic signal control/deep reinforcement learning/Dueling double DQN/Dueling network引用本文复制引用
基金项目
浙江省自然科学基金项目(LTGS23F030002)
嘉兴市应用性基础研究项目(2023AY11034)
浙江省尖兵领雁研发攻关计划项目(2023C01174)
国家自然科学基金项目(61603154)
浙江省自然科学基金项目(LY19F030014)
工业控制技术国家重点实验室开放课题(ICT2022B52)
出版年
2024
NETL
NSTL
万方数据