Echo State Network Based on Improved Knowledge Distillation for Edge Intelligence

扫码查看

原文链接

万方数据

外文摘要：Echo state network(ESN)as a novel artificial neural network has drawn much attention from time series prediction in edge intelligence.ESN is slightly insufficient in long-term memory,thereby impacting the predic-tion performance.It suffers from a higher computational overhead when deploying on edge devices.We firstly intro-duce the knowledge distillation into the reservoir structure optimization,and then propose the echo state network based on improved knowledge distillation(ESN-IKD)for edge intelligence to improve the prediction performance and reduce the computational overhead.The model of ESN-IKD is constructed with the classic ESN as a student net-work,the long and short-term memory network as a teacher network,and the ESN with double loop reservoir struc-ture as an assistant network.The student network learns the long-term memory capability of the teacher network with the help of the assistant network.The training algorithm of ESN-IKD is proposed to correct the learning direc-tion through the assistant network and eliminate the redundant knowledge through the iterative pruning.It can solve the problems of error learning and redundant learning in the traditional knowledge distillation process.Extensive ex-perimental simulation shows that ESN-IKD has a good time series prediction performance in both long-term and short-term memory,and achieves a lower computational overhead.

外文关键词：

Echo state networkReservoir structure optimizationKnowledge distillationEdge intelligenceTime series prediction

作者：

Jian ZHOU、Yuwen JIANG、Lijie XU、Lu ZHAO、Fu XIAO

展开 >

作者单位：

College of Computer,Nanjing University of Posts and Telecommunications,Nanjing 210003,China

Jiangsu High Technology Research Key Laboratory for Wireless Sensor Networks,Nanjing 210003,China

基金：

National Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaScience and Technology Planning Project of Jiangsu Province1311 Talent Program of Nanjing University of Posts and Telecommunications

项目编号：

619722106180220661803212BE2020729

出版年：

2024

DOI：

10.23919/cje.2022.00.292

电子学报(英文)

CSTPCDEI

ISSN：1022-4653

年,卷(期)：2024.33(1)

参考文献量40