Collision prevention decision-making method for autonomous driving based on CQL-SAC algorithm
In response to the problems of value function overestimation,low learning efficiency,and poor safety in deep reinforcement learning for autonomous driving tasks,a collision avoidance decision-making method was proposed.Firstly,by integrating conservative Q-learning(CQL)algorithm with soft actor-critic(SAC)algorithm,the CQL-SAC algorithm was proposed to alleviate the problem of value overestimation.Then,expert experience was introduced during the algorithm training process to achieve fast convergence and solve the problem of low learning efficiency.Finally,the collision prevention module was used to perform safety checks and corrections on the actions output by the CQL-SAC algorithm,in order to avoid vehicle collisions.The effectiveness of this scheme was verified in a simulation scenario based on highways.The simulation results show that during the training phase,the CQL-SAC algorithm improves the convergence speed by 12.5％and 5.4％compared with the SAC algorithm and in-sample actor-critic(InAC)algorithm,respectively;and the algorithm convergence speed is further improved by 14.3％after introducing expert experience.During the testing phase,the proposed scheme shows better performance with a success rate increase of 17 and 12 percentage points and an average turn reward increase of 23.1％and 10.7％compared with the SAC and InAC algorithms,respectively.
万方数据
维普
