首页|基于深度学习与卡尔曼滤波的多模态融合里程计

基于深度学习与卡尔曼滤波的多模态融合里程计

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
里程计是同步定位与建图技术的重要组成部分,现有里程计算法多以视觉或激光传感器为主,未能充分发挥多模态传感器的优势,在特征缺失场景和复杂环境下鲁棒性不足.针对此问题,本文同时采用了激光雷达、彩色相机和惯性测量单元等多模态传感器的数据,提出一种多模态融合深度网络——MLVIO-Net,并与一个误差状态卡尔曼滤波器(ESKF)共同组成多模态融合里程计.MLVIO-Net由特征金字塔网络、多层双向长短期记忆(Bi-LSTM)网络、位姿估计网络和位姿优化网络组成,实现了对多模态数据的紧密融合.特征金字塔网络可以对激光点云进行层级化特征提取,LSTM网络可以更好地学习惯性测量数据的时序特征,位姿估计网络和位姿优化网络可以逐级优化预测结果.ESKF通过惯性测量单元运动学模型预测位姿,并利用来自MLVIO-Net的预测结果校正位姿,在提升预测精度的同时,还可以大幅提高里程计的输出帧率.在开源数据集KITTI上进行验证,实验和测试结果表明,与其他常见的算法相比,本文提出的多模态融合里程计具有更高的精度.
Multimodal Fusion Odometer Based on Deep Learning and Kalman Filter
Odometry is an important component of simultaneous localization and mapping(SLAM)technology.Existing odometry algorithms mainly rely on visual or laser sensors,failing to fully exploit the advantages of multimodal sensors and exhibiting insufficient robustness in feature-deprived scenarios and complex environments.To address this issue,this paper utilizes data from multimodal sensors including lidar,color camera,and inertial measurement unit,and proposes a multimodal fusion deep network,MLVIO-Net,which collaborates with an error state Kalman filter(ESKF)to form a multimodal fusion odometry system.MLVIO-Net consists of a feature pyramid network,multi-layer bidirectional long-short term memory(Bi-LSTM)network,pose estimation network,and pose optimization network,achieving close integration of multimodal data.The feature pyramid network performs hierarchical feature extraction on lidar point clouds,while the LSTM network effectively learns the temporal features of inertial measurement data.The pose estimation and optimization networks iteratively refine the predicted results.The ESKF predicts poses using the kinematic model of the inertial measurement unit and corrects poses using the predictions from MLVIO-Net,thereby improving prediction accuracy and significantly enhancing the output frame rate of the odometry.Experimental results on the open dataset KITTI demonstrate that the proposed multimodal fusion odometry exhibits higher accuracy and robustness compared to other common algorithms.

remote sensinglidarsimultaneous localization and mappingmultimodal fusion odometererror state Kalman filterodometer

李隆、安毅、谢丽蓉、孙卓、董宏翔

展开 >

新疆大学电气工程学院,新疆 乌鲁木齐 830046

大连理工大学控制科学与工程学院,辽宁 大连 116024

遥感 激光雷达 同步定位与建图 多模态融合里程计 误差状态卡尔曼滤波 里程计

国家自然科学基金辽宁省自然科学基金计划项目山西省科技重大专项揭榜项目新疆维吾尔自治区重点研发项目

621730552023-MS-093201911010142022B02038

2024

10.3788/LOP232559

激光与光电子学进展
中国科学院上海光学精密机械研究所

激光与光电子学进展

CSTPCD北大核心
影响因子：1.153
ISSN：1006-4125
年,卷(期)：2024.61(18)
  • 4