基于混合智能的街景影像知识提取方法
Research on knowledge extraction from street scene images based on hy-brid intelligence
刘万增 1陈杭 2任加新 3张兆江 4李然 1赵婷婷 1翟曦 1朱秀丽1
作者信息
- 1. 国家基础地理信息中心,北京 100830;自然资源部时空信息与智能服务重点实验室,北京 100830;湖北珞珈实验室,湖北武汉 430079
- 2. 自然资源部时空信息与智能服务重点实验室,北京 100830;河北工程大学矿业与测绘工程学院,河北邯郸 056038
- 3. 自然资源部时空信息与智能服务重点实验室,北京 100830;中南大学地球科学与信息物理学院,湖南长沙 410083
- 4. 河北工程大学矿业与测绘工程学院,河北邯郸 056038
- 折叠
摘要
针对街景影像目标的智能化提取难题,本文提出了一种基于混合智能的街景影像知识提取方法(K-CAPSNet).首先,在现有全景分割网络的基础上,同时关注街景影像的通道信息和空间信息,发展了一种联合注意力机制的全景分割网络,以提高目标分割精度;其次,将人们在生产、生活中形成的街景知识融入街景影像认知过程,借助先验知识设置目标标记阈值,对分割结果进行优化;然后,进一步根据街景影像先验知识验证街景目标之间的拓扑关系并利用深度信息进行空间关系知识挖掘;最后,采用语义模板对街景目标类型、数量及空间关系进行描述和表达.试验表明,相较于基线网络,本文方法在全景分割质量和识别质量方面都有明显提升,较好地实现了对街景影像知识的提取与表达.
Abstract
This study presents a hybrid intelligence-based approach,named K-CAPSNet,for extracting knowledge from streetscape images.To tackle the challenge of intelligent extraction of streetscape image objects,we develop a panoramic seg-mentation network with a joint attention mechanism that integrates both channel information and spatial information of streetscape images.This improves the object segmentation accuracy.Additionally,we incorporate streetscape knowledge,which is formed by people in production and life,into the streetscape image cognition process.We set the object marking threshold using a priori knowledge to optimize the segmentation results.Moreover,we utilize the a priori knowledge of streetscape images to verify the topological relationship between streetscape objects and to mine spatial relationship knowledge using depth information.Finally,we employ semantic templates to describe and express the type,number,and spatial rela-tionship between streetscape objects.The experimental results demonstrate that our method outperforms the baseline network and significantly improves the quality of panoramic segmentation and recognition,thereby achieving better extraction and ex-pression of the knowledge of streetscape images.
关键词
混合智能/先验知识/全景分割/场景认知/注意力机制/空间关系Key words
hybrid intelligence/prior knowledge/panoptic segmentation/scene cognition/attentional mechanisms/spatial re-lationships引用本文复制引用
基金项目
国家自然科学基金(42394062)
国家重点研发计划(2022YFB3904205)
湖北珞珈实验室开放基金资助项目(220100037)
出版年
2024
NETL
NSTL
万方数据