首页|基于柔性演员-评论家算法的决策规划协同研究

基于柔性演员-评论家算法的决策规划协同研究

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
  • 维普
为了解决基于常规深度强化学习(Deep Reinforcement Learning,DRL)的自动驾驶决策存在学习速度慢、安全性及合理性较差的问题,本文提出一种基于柔性演员-评论家(Soft Actor-Critic,SAC)算法的自动驾驶决策规划协同方法,并将SAC算法与基于规则的决策规划方法相结合设计自动驾驶决策规划协同智能体.结合自注意力机制(Self Attention Mechanism,SAM)和门控循环单元(Gate Recurrent Unit,GRU)构建预处理网络;根据规划模块的具体实现方式设计动作空间;运用信息反馈思想设计奖励函数,给智能体添加车辆行驶条件约束,并将轨迹信息传递给决策模块,实现决策规划的信息协同.在CARLA自动驾驶仿真平台中搭建交通场景对智能体进行训练,并在不同场景中将所提出的决策规划协同方法与常规的基于SAC算法的决策规划方法进行比较,结果表明,本文所设计的自动驾驶决策规划协同智能体学习速度提高了25.10%,由其决策结果生成的平均车速更高,车速变化率更小,更接近道路期望车速,路径长度与曲率变化率更小.
Collaborative Study of Decision-making and Trajectory Planning for Autonomous Driving Based on Soft Acto-Critic Algorithm
To improve the learning speed,safety and rationality of autonomous driving decision-making,this paper proposed a collaborative method of autonomous driving decision-making and planning based on Soft Actor-Critic(SAC)algorithm.The autonomous driving decision planning collaborative agent was designed by introducing the SAC algorithm with the rule-based decision planning method.Combined with the Self-Attention Mechanism(SAM)and the Gated Recurrent Unit(GRU),a preprocessing network was constructed to improve the agent's ability to understand traffic scenarios and improve the agent's learning speed.Considering the specific implementation mode of the planning module,the study used the action space to improve the executability of the decision-making results.The reward function was designed by using the information feedback,adding the constraints of vehicle driving conditions to the agent,and transmitting the trajectory information to the decision-making module.The collaboration of decision-making and planning improved the safety and rationality of decision-making.The dynamic traffic scenarios were built in the CARLA autonomous driving simulation platform to train the agent,and the proposed decision-making and planning collaboration method was compared with the conventional decision-making planning method based on SAC algorithm in different scenarios.The experimental results showed that the learning speed of the agent designed in this paper had increased by 25.10%.The average vehicle speed generated by its decision outcomes was higher and closer to the expected road speed.The speed variation rate produced by its decision outcomes was smaller,and the path length and curvature variation rate resulting from its decision outcomes were also smaller compared to traditional methods.

intelligent transportationautonomous drivingsoft actor-critic algorithmcollaborative decision and planningdeep reinforcement learning

唐斌、刘光耀、江浩斌、田宁、米伟、王春宏

展开 >

江苏大学,汽车工程研究院,江苏镇江 212013

江苏罡阳转向系统有限公司,江苏泰州 225318

智能交通 自动驾驶 柔性演员-评论家算法 决策规划协同 深度强化学习

国家自然科学基金江苏省六大人才高峰项目泰州市科技支撑计划项目

522252122019-GDZB-084TG202307

2024

10.16097/j.cnki.1009-6744.2024.02.011

交通运输系统工程与信息
中国系统工程学会

交通运输系统工程与信息

CSTPCD北大核心
影响因子：0.664
ISSN：1009-6744
年,卷(期)：2024.24(2)
  • 12