- source
- (Gauthier et al. 2021)
- tags
- Reservoir computing
Summary
This paper bases itself on the demonstration that some reservoir computers (echo-state networks) are mathematically identical to nonlinear vector autoregression (NVAR) machines (Bollt 2021). A NVAR is just a regression over a feature vector composed of \(k\) time-delay observations of the dynamical system to be learned and nonlinear functions of these observations.
The authors introduce Next-Generation Reservoir computing (NG-RC) which is essentially a NVAR. Instead of the standard RC setup, they regress the outer layer over a vector
\[\mathbb{O}_{\text{total} }= c + \mathbb{O}_{\text{lin}} + \mathbb{O}_{\text{nonlin}}\]
where \(c\) is a constant, \(\mathbb{O}_{\text{lin}}\) is the vector of time delayed input observations and \(\mathbb{O}_{\text{nonlin}}\) is a vector of non-linear transformations of these observations.
Then, the output layer computes the NVAR output as a linear transformation of the feature vector, through \(\mathbf{Y}_i = \mathbf{W}_{\text{out}} \mathbb{O}_{\text{total}, i}\).
For a training dataset \(\mathbf{Y}_d\), output \(\mathbf{Y}\) is matched to it by solving a least-square linear regression problem.
The authors then evaluate the NG-RC model on two chaotic dynamical systems and show some performance scores and a few train/test trajectories comparisons.
Comments
There doesn’t seem to be any comparison against standard RC, which is what this model is supposed to replace.
The benefits of reservoir computing probably lie beyond echo-state networks. With different reservoirs, one can harness the computations of complex non-linear dynamical systems which might not be possible using only recurrent neural networks.
Bibliography
- Daniel J. Gauthier, Erik Bollt, Aaron Griffith, Wendson A. S. Barbosa. . "Next Generation Reservoir Computing". Nature Communications 12 (1). Nature Publishing Group:5564. DOI.
- Erik Bollt. . "On Explaining the Surprising Success of Reservoir Computing Forecaster of Chaos? the Universal Machine Learning Dynamical System with Contrast to VAR and DMD". Chaos (woodbury, N.Y.) 31 (1):013108. DOI.