首页|基于样本优化的PPO算法在单路口信号控制的应用

基于样本优化的PPO算法在单路口信号控制的应用

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
优化交通信号的控制策略可以提高道路车辆通行效率,缓解交通拥堵.针对基于值函数的深度强化学习算法难以高效优化单路口信号控制策略的问题,构建了一种基于样本优化的近端策略优化(MPPO)算法的单路口信号控制方法,通过对传统PPO算法中代理目标函数进行最大化提取,有效提高了模型选择样本的质量,采用多维交通状态向量作为模型观测值的输入方法,以及时跟踪并利用道路交通状态的动态变化过程.为了验证MPPO算法模型的准确性和有效性,在城市交通微观模拟软件(SUMO)上与值函数强化学习控制方法进行对比.仿真实验表明,相比于值函数强化学习控制方法,该方法更贴近真实的交通场景,显著加快了车辆累计等待时间的收敛速度,车辆的平均队列长度和平均等待时间明显缩短,有效提高了单路口车辆的通行效率.
Application of Sample-optimized PPO Algorithm in Single Intersection Signal Control
Optimizing the control strategy of traffic signals can improve the efficiency of vehicular traffic on roads and alleviate congestion.To overcome the challenge of efficiently optimizing signal control strategies at single intersections using value function-based deep reinforcement learning algorithms,this study develops a method based on sample optimization called modified proximal policy optimization(MPPO).This approach enhances the quality of model sample selection by maximizing the extraction from the agent target function in the traditional PPO algorithm.It employs a multi-dimensional traffic state vector as input for the model's observations,enabling it to promptly track and utilize the dynamic changes in road traffic conditions.The accuracy and effectiveness of the MPPO algorithm model are verified by comparing it with value function reinforcement learning control methods using the urban traffic micro simulation software(SUMO).Simulation experiments show that this approach closely resembles real traffic scenarios compared to value function reinforcement learning control methods.It significantly accelerates the convergence speed of cumulative vehicle waiting time,noticeably reduces the average vehicle queue length and waiting time,and effectively improves the traffic throughput at the intersection.

traffic signal controldeep reinforcement learningproximal policy optimization(PPO)algorithmsurrogate objective functionstate feature vector

张国有、张新武

展开 >

太原科技大学计算机科学与技术学院,太原 030024

交通信号控制 深度强化学习 近端策略优化算法 代理目标函数 状态特征向量

国家自然科学基金山西省自然科学基金太原科技大学科技创新基金山西省基础研究计划

6207232520220302122114520212039202103021224272

2024

10.15888/j.cnki.csa.009544

计算机系统应用
中国科学院软件研究所

计算机系统应用

CSTPCD
影响因子：0.449
ISSN：1003-3254
年,卷(期)：2024.33(6)