空间自回归模型下不完整大数据缺失值插补算法
Interpolation Algorithm for Missing Values of Incomplete Big Data in Spatial Autoregressive Model
刘晓燕 1翟建国1
作者信息
- 1. 昆明理工大学信息工程与自动化学院,昆明 650504
- 折叠
摘要
针对不完整大数据因其自身结构具有不规则性,导致在进行缺失值插补时计算量大、插补精度低的问题,提出空间自回归模型下不完整大数据缺失值插补算法.利用迁移学习算法在动态权重下过滤出原始数据中冗余数据,区分异常和正常数据,提取残缺数据,采用最小二乘回归对残缺数据实施修补.将缺失值插补分为3种类型,分别为一阶空间自回归模型插补、空间自回归模型插补和多重插补法.根据实际情况将修补后数据插补到合适的位置,实现不完整大数据缺失值插补.实验结果表明,所提方法具有良好的缺失值插补能力.
Abstract
Incomplete big data,due to its irregular structure,has a large amount of computation and low interpolation accuracy when interpolation misses values.Therefore,a missing value interpolation algorithm for incomplete big data based on spatial autoregressive model is proposed.Using a migration learning algorithm to filter out redundant data from the original data under dynamic weights,to distinguish abnormal data from normal data,and to extract incomplete data.Using least square regression to repair the incomplete data.The missing value interpolation is divided into three types,namely,first order spatial autoregressive model interpolation,spatial autoregressive model interpolation,and multiple interpolation.The repaired data is interpolated to the appropriate location according to the actual situation,implementing incomplete big data missing value interpolation.Experimental results show that the proposed method has good interpolation ability for missing values.
关键词
迁移学习/不完整大数据/缺失值插补/空间回归模型/数据修正Key words
transfer learning/incomplete big data/imputation of missing values/spatial regression model/data correction引用本文复制引用
基金项目
云南省自然科学基金(202224143456)
出版年
2024
国家科技期刊平台
NSTL
维普
万方数据