Bayesian Inverse Reinforcement Learning-based Reward Learning for Automated Driving
Studying driving policies with wide-ranging scenario adaptability is crucial to realizing safe,efficient,and harmonious automated driving.Deep reinforcement learning has shown great potential in driving policy learning with its excellent function approximation and representation capabilities.However,it is extremely challenging to design a reward function suitable for various complex driving scenarios,and driving strategies'generalization ability needs to be urgently improved.Aiming at the difficulty in designing the reward function,an approximate likelihood model of human drivers'driving policy is built considering their preferences and a method of learning an approximate posterior distribution over the reward function through sparse action sampling based on curve interpolation and approximate variational inference is proposed,resulting in a Bayesian neural network.Tackling the wrong rewards originate from the uncertainty of a reward function,an uncertainty-aware human-like driving policy learning method based on the posterior distribution over the reward function is proposed,which maximizes the reward while penalizing the epistemic uncertainty.The proposed methods are validated in simulated highway and urban driving scenarios in the NGSIM US-101 and nuPlan datasets.The results show that the proposed method overcomes the problem of poor performance of the reward function based on the linear combination of hand-crafted state features,models the uncertainty of the reward function,and improves the generalization ability of the reward function to high-dimensional nonlinear problems.The learned reward function and the learning stability are significantly better than the mainstream inverse reinforcement learning method.Moreover,penalizing the uncertainty of the reward function improves the human likeness and safety of the driving policy and the training stability.The proposed uncertainty-aware human-like driving policy significantly outperforms the driving policies based on behavior cloning and maximum entropy inverse reinforcement learning.
NETL
NSTL
万方数据
