Adaptive Nonlinear System Identification with Echo State Networks

Part of Advances in Neural Information Processing Systems 15 (NIPS 2002)

Bibtex Metadata Paper

Authors

Herbert Jaeger

Abstract

Echo state networks (ESN) are a novel approach to recurrent neu(cid:173) ral network training. An ESN consists of a large, fixed, recurrent "reservoir" network, from which the desired output is obtained by training suitable output connection weights. Determination of op(cid:173) timal output weights becomes a linear, uniquely solvable task of MSE minimization. This article reviews the basic ideas and de(cid:173) scribes an online adaptation scheme based on the RLS algorithm known from adaptive linear systems. As an example, a 10-th or(cid:173) der NARMA system is adaptively identified. The known benefits of the RLS algorithms carryover from linear systems to nonlinear ones; specifically, the convergence rate and misadjustment can be determined at design time.