基于哈希算法的互联网平台数据中台资源检索方法
梁艳春 1阮宜龙 2李晨阳 3张宏俊3
作者信息
- 1. 中国通信建设集团有限公司 北京 丰台 100071
- 2. 中国电信集团有限公司 北京 100032
- 3. 中国通信服务股份有限公司 北京 丰台 100071
- 折叠
摘要
由于检索请求数据自身具有高维特征,导致检索输出的查准率和查全率偏低,为此,本文提出基于哈希算法的互联网平台数据中台资源检索方法.以信息跨域检索为导向,借助哈希算法实现对输入互联网平台数据中台资源检索请求的降维处理,在对输入数据进行清洗、去重、分词等预处理操作的基础上,使用词袋模型的方法,将文本转化为向量,再借助主成分分析法实现对向量的降维.在检索阶段,将与检索请求相似度最高(欧氏距离最小的)资源作为最终的检索输出结果.在测试结果中,资源检索方法面对不同类型的资源检索请求,对应的查准率稳定在91.0%以上,查全率稳定在90.0%以上.
Abstract
Due to the high-dimensional characteristics of the retrieval request data itself,the precision and recall of the retrieval output are low.Therefore,this article proposes a method for searching middle platform resources in internet platform data based on hash algorithm.Guided by cross domain information retrieval,a hash algorithm is used to reduce the dimensionality of resource retrieval requests for input internet platform data.Based on preprocessing operations such as cleaning,deduplication,and word segmentation of the input data,a word bag model is used to convert the text into vectors,and then principal component analysis is used to reduce the dimensionality of the vectors.In the retrieval stage,the resource with the highest similarity(minimum Euclidean distance)to the retrieval request will be used as the final retrieval output result.In the test results,the resource retrieval method faces different types of resource retrieval requests,and the corresponding precision is stable at over 91.0%,while the recall is stable at over 90.0%.
关键词
哈希算法/互联网平台/数据中台/资源检索/信息跨域检索/降维处理/词袋模型/主成分分析法/欧氏距离Key words
Hash algorithm/Internet platform/Data center/Resource retrieval/Cross domain information retrieval/Dimension reduction treatment/Word bag model/Principal component analysis method/Euclidean distance引用本文复制引用
基金项目
江苏省研究生科研与实践创新计划(KYCX22_1019)
出版年
2024
NETL
NSTL
万方数据