Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes

Mathieu N Galtier; Camille Marini; Gilles Wainrib; Herbert Jaeger

doi:10.1016/j.neunet.2014.04.002

Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes

Neural Netw. 2014 Aug:56:10-21. doi: 10.1016/j.neunet.2014.04.002. Epub 2014 Apr 26.

Authors

Mathieu N Galtier¹, Camille Marini², Gilles Wainrib³, Herbert Jaeger⁴

Affiliations

¹ School of Engineering and Science, Jacobs University Bremen gGmbH, 28759 Bremen, Germany. Electronic address: mathieu.galtier@inria.fr.
² Institut für Meereskunde, Zentrum für Meeres- und Klimaforschung, Universität Hamburg, Hamburg, Germany; MINES ParisTech, 1, rue Claude Daunesse, F-06904 Sophia Antipolis Cedex, France.
³ Laboratoire Analyse Géométrie et Applications, Université Paris XIII, France.
⁴ School of Engineering and Science, Jacobs University Bremen gGmbH, 28759 Bremen, Germany.

PMID: 24815743
DOI: 10.1016/j.neunet.2014.04.002

Abstract

A method is provided for designing and training noise-driven recurrent neural networks as models of stochastic processes. The method unifies and generalizes two known separate modeling approaches, Echo State Networks (ESN) and Linear Inverse Modeling (LIM), under the common principle of relative entropy minimization. The power of the new method is demonstrated on a stochastic approximation of the El Niño phenomenon studied in climate research.

Keywords: Echo state networks; El Nino southern oscillation; Linear inverse modeling; Relative entropy; Stochastic processes.

Publication types

Comparative Study
Research Support, Non-U.S. Gov't

MeSH terms

Algorithms
Computer Simulation
El Nino-Southern Oscillation
Entropy*
Linear Models
Neural Networks, Computer*
Nonlinear Dynamics*
Stochastic Processes*
Time