首页|融合AP聚类算法和宽度学习系统的分布外硬盘故障预测

融合AP聚类算法和宽度学习系统的分布外硬盘故障预测

扫码查看
原文链接
  • 万方数据
  • 维普
硬盘是云数据中心最主要的存储设备,硬盘故障预测是保障数据安全的重要手段.但是,硬盘的故障与健康样本之间存在着极端的数量不平衡问题,这会导致模型偏差;此外,不同型号的硬盘数据分布存在一定的差异,在特定硬盘数据上训练的模型往往不适用于其他硬盘.对于这两个问题,文中提出了一种融合AP聚类算法和宽度学习系统的分布外硬盘故障预测方法.针对样本不平衡问题,文中使用AP聚类算法对硬盘故障出现前一阶段的样本集进行聚类,将与故障样本处于同一聚类簇的样本扩充为故障样本.针对不同型号硬盘分布存在差异的问题,文中结合流形正则化框架和宽度学习系统来学习硬盘数据的低维结构,提高模型对未知分布数据的泛化能力.实验结果表明,在AP聚类算法重采样的样本集上,相较于用于对比的重采样方法得到的样本集,多种故障预测方法的F1_Score取得了平均0.2的提升.此外,在分布外硬盘故障预测任务上,所提模型的F1_Score相比对比方法提升了 0.1～0.2.
Out-of-Distribution Hard Disk Failure Prediction with Affinity Propagation Clustering and Broad Learning Systems
Hard disk is the primary storage device in cloud data centers,and hard disk failure prediction is crucial for ensuring da-ta security.However,there is a significant imbalance between failure and healthy SMART samples,which can lead to model bias.Moreover,hard disk models exhibit varying data distributions.Prediction models trained on specific hard disk data may not be suitable for other hard disks.To address these issues,this paper proposes a method for out-of-distribution hard disk failure pre-diction by combining the AP clustering algorithm and the broad learning system.To tackle the sample imbalance problem,this pa-per uses the AP clustering algorithm to cluster samples close to failure and treats all samples in the cluster containing determined failure instances as additional failure samples.To address the distribution differences of hard disk models,this paper combines the manifold regularization framework and the broad learning system to learn the low-dimensional structure of hard disk data,thereby improving the model's generalization ability to unknown data.Experimental results show that,on the dataset resampled by the AP clustering algorithm,the F1_Score of multiple methods increases by an average of 0.2 compared to the datasets resampled by comparative methods.Additionally,in the task of predicting out-of-distribution hard disk failures,the F1_Score of the proposed model increases by 0.1～0.2 compared to other methods.

Hard disk failure predictionClass imbalanceOut-of-distribution generalizationAffinity propagation clusteringBroad learning systemManifold learning

王屹阳、刘发贵、彭玲霞、钟国祥

展开 >

华南理工大学计算机科学与工程学院 广州 510006

鹏城实验室 广东深圳 518066

硬盘故障预测 类不平衡 分布外泛化 AP聚类 宽度学习系统 流形学习

鹏城实验室重大项目广东省基础与应用基础研究重大项目广州市重点领域研发计划项目广东省省级科技计划项目

PCL2023A092019B0303020022020070300062021B1111600001

2024

10.11896/jsjkx.230600103

计算机科学
重庆西南信息有限公司（原科技部西南信息中心）

计算机科学

CSTPCD北大核心
影响因子：0.944
ISSN：1002-137X
年,卷(期)：2024.51(8)
  • 3