Information-theoretic-based interpretable multimodal perception for intelligent vehicles
Intelligent driving has become one of the most compelling topics of interest.Nevertheless,current intelligent driving technologies still face challenges,such as missed detection due to large vehicle occlusion and false detection caused by sensor accuracy degradation in sudden light changes.Multimodal perception technology for intelligent vehicles has emerged to ensure the safety of vehicle perception in complex scenarios.However,the existing multimodal fusion methods are still limited to the improvement of detection accuracy,lack of interpretability of the perception process and lack of evaluation indexes for the model perception process.In this paper,from the information theory perspective,we design the perception model according to the communication model.We propose a multimodal fusion perception model based on joint source-channel coding theory to explain the perception process of the model theoretically.At the same time,we propose a new evaluation index,average entropy variation(AEV),which is used to reflect the stability of the model during its perceptual interaction with the outside world in real time.Further,the perceptual process is quantified and analyzed to increase the interpretability of the model.Finally,we compare the evaluation results with other advanced perceptual models in the KITTI dataset,and our model decreases the average entropy variation to 0.5904,which better ensures the perceptual safety of the detection task.
interpretabilitytheory of informationjoint source-channel codingmultimodal fusionintelligent driving
NETL
NSTL
万方数据
