1. Ghent University An overview of Reservoir Computing: theory, applications and implementations Benjamin Schrauwen David Verstraeten and Jan Van Campenhout Electronics and Information Systems Department Ghent University – Belgium April 27 2007 – ESANN

2. 2/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Intro In ML and pattern recognition: mostly FF structures NN, Bayesian models, kernel methods … Well understood, non-temporal Many applications: temporal domain Time series prediction Financial data Dynamic systems and control Robotics Vision, speech,… Takens’ theorem: explicit embedding Or introducing recurrence: loopy belief propagation, RNN

3. 3/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Intro Recurrent Neural Networks: Hopfield (1982): Specific topologies with symmetric weights Information stored in attractors Werbos (1974): BackProp Through Time (and all its improvements) Problem of fading gradient, mathematically difficult Few applications, difficult to master Special topologies: LSTM (Schmidhuber) RNN are universal approximators (ESANN special session 2005)

4. 4/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Intro Recurrent structures without the training: Reservoir Computing Early related work by Buonomano (1995) and Laurenco (1994) Independently discovered: Jaeger (2001): Echo State Networks (engineering) Maass (2002): Liquid State Machines (neuroscience) Shortly afterwards: Steil (2003): weight dynamics of Atiya-Parlos equivalent RC: Fixed (random) topology operated in correct dynamic regime Linear “readout” function which is trained

5. 5/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Reservoir Computing

6. 6/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Reservoir Computing Properties of reservoir: Exact topology, connectivity, weights: not important Has to have fading memory: when not chaotic Longest memory if at the edge of stability: memory = nr. nodes Reservoir size can be large: no over-fitting Training with linear regression (pseudo-inv, ridge regression): No local minima, no problems with recurrent structure, one shot learning Can do regression, classification, prediction On-line learning also possible with LMS and RLS

7. 7/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Reservoir Computing RC does on-line computing: prediction at every time-step Theoretically: Any time-invariant filter with fading memory can be learned But: unable to implement generic FSMs Recently Maass (2006): when adding output feedback Also non-fading memory filters: generic FSMs Ability to simulate any n-th order dynamical system Turing universal

8. 8/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 RC: tone generation example Taken from H. Jaeger

9. 9/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Usual setup and training Create random weight matrices Rescale reservoir weights so that max absolute eigenvalue close to one (edge of stability) Excite reservoir with input and record all states Train readouts by minimizing (Aw-b) 2 A space time w B

10. 10/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Influence of parameters Not very important: Connection fraction Exact topology Weight distribution timescale error dynamic regime chaos error reservoir size error

11. 11/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 A useful ESANN analogy Input Compute output Reservoir Reservoir state 2008

12. 12/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 State space view

13. 13/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 z' * * * * * * * ● ● ● ● ● ● x y * * * * * * * x' y' Kernel ● ● ● ● ● ● Kernel: Projection of input space into high- dimensional feature space. ● Conventional methods rely on 'kernel trick' to avoid explicitly going to feature space. ● Reservoir computing works in feature space, but reservoir state contains temporal information. Temporal to spatial transformation Link to kernel machines

14. 14/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Link to FSMs FSMRC with output feedback

15. 15/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 RC: Applications Chaotic time series prediction: order of magnitude better than SOA Speech recognition on small vocabulary: outperform HMM-based recognizer (Sphinx) Digits recognition: better than SOA Robot control System identification Noise removal/modelling …

16. 16/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Larger example: speech ReadoutReservoir Pre-processing Speech Post-processing Σ Σ... Downsampling... t t t Reservoir state Mean... WTA '6'

17. 17/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 RC: novel computing paradigm RC presents a novel way of looking at computation “Random” dynamic systems can be used by only training a linear readout layer RC already used to show general computing capabilities of: Microcolumn structure in the cortex Gene regulatory network The visual cortex of a real cat Implementation: Toolbox (freely available at http://www.elis.ugent.be/rct) “Bucket of water”, aVLSI, digital hardware Photonics (in progress)

18. 18/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 Current research topics Theoretical: Proper understanding of importance of dynamics Regularisation Reservoir optimisation Intrinsic plasticity: unsupervised reservoir adaptation based on infomax to set dynamic regime Timescales Modular reservoirs Generic reservoir idea Applications

19. 19/19 An overview of Reservoir Computing: theory, applications and implementations ESANN - April 27 2007 This session Reservoir optimisation Intrinsic plasticity: Steil, Verstraeten et al., Wardermann et al. Reservoir pruning: Dutoit et al. Alternate reservoir ideas Gao et al. Lourenco