首页|Robust echo state network with sparse online learning
Robust echo state network with sparse online learning
扫码查看
点击上方二维码区域，可以放大扫码查看
原文链接
NSTL
Elsevier
Echo state network (ESN) is an effective tool for nonlinear systems modeling. To handle irregular noises or outliers in practical systems and alleviate the overfitting issue, the robust echo state network with sparse online learning (RESN-SOL) is proposed. Firstly, the epsilon-insensitive loss function is introduced to replace the commonly used quadratic loss function, which is theoretically optimal for Gaussian noise distribution. Secondly, the online gradient descent algorithm is used to calculate the network readout. Notably, the better learning performance can be achieved by the constant learning rate rather than the decreasing step size. Based on this observation, the sparse online learning algorithm (SOL) is proposed, in which the constant step size is used. Particularly, the SOL is able to truncate the small weights in network readout to zero for achieving sparsity. Furthermore, the convergence of RESN-SOL is theoretically analyzed, which implies the tradeoff between learning performance and readout sparsity can be controlled by a predefined sparsity parameter. Finally, the proposed method is verified in two simulated benchmarks and an actual dynamical wastewater treatment system. Experimental results demonstrate that the RESN-SOL exhibits better robustness against outliers, network compactness and modeling accuracy than other existing algorithms. (C) 2022 Elsevier Inc. All rights reserved.
Robust echo state networksepsilon-Insensitive loss functionSparse online learningNonlinear systems modelingWastewater treatment systemNONLINEAR-SYSTEMSNEURAL-NETWORKOPTIMIZATIONPREDICTIONDESIGNFINITE
NSTL
Elsevier
