Autonomous Optimization of Targeted Stimulation of Neuronal Networks

PLoS Comput Biol. 2016 Aug 10;12(8):e1005054. doi: 10.1371/journal.pcbi.1005054. eCollection 2016 Aug.

Abstract

Driven by clinical needs and progress in neurotechnology, targeted interaction with neuronal networks is of increasing importance. Yet, the dynamics of interaction between intrinsic ongoing activity in neuronal networks and their response to stimulation is unknown. Nonetheless, electrical stimulation of the brain is increasingly explored as a therapeutic strategy and as a means to artificially inject information into neural circuits. Strategies using regular or event-triggered fixed stimuli discount the influence of ongoing neuronal activity on the stimulation outcome and are therefore not optimal to induce specific responses reliably. Yet, without suitable mechanistic models, it is hardly possible to optimize such interactions, in particular when desired response features are network-dependent and are initially unknown. In this proof-of-principle study, we present an experimental paradigm using reinforcement-learning (RL) to optimize stimulus settings autonomously and evaluate the learned control strategy using phenomenological models. We asked how to (1) capture the interaction of ongoing network activity, electrical stimulation and evoked responses in a quantifiable 'state' to formulate a well-posed control problem, (2) find the optimal state for stimulation, and (3) evaluate the quality of the solution found. Electrical stimulation of generic neuronal networks grown from rat cortical tissue in vitro evoked bursts of action potentials (responses). We show that the dynamic interplay of their magnitudes and the probability to be intercepted by spontaneous events defines a trade-off scenario with a network-specific unique optimal latency maximizing stimulus efficacy. An RL controller was set to find this optimum autonomously. Across networks, stimulation efficacy increased in 90% of the sessions after learning and learned latencies strongly agreed with those predicted from open-loop experiments. Our results show that autonomous techniques can exploit quantitative relationships underlying activity-response interaction in biological neuronal networks to choose optimal actions. Simple phenomenological models can be useful to validate the quality of the resulting controllers.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Animals
  • Brain* / physiology
  • Brain* / radiation effects
  • Computational Biology
  • Electric Stimulation*
  • Machine Learning
  • Models, Neurological*
  • Nerve Net* / physiology
  • Nerve Net* / radiation effects
  • Rats

Grants and funding

This work was supported by the Bundesministerium für Bildung und Forschung, Germany (https://www.bmbf.de, FKZ 01GQ0830 - Bernstein Focus Neurotechnology Freiburg-Tübingen; UE) and by the German Research Foundation through the Brain-Links-BrainTools Cluster of Excellence (DFG, EXC 1086, www.dfg.de; UE, MR). Article processing charges were funded by the DFG and the University of Freiburg (Open Access Publishing program). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.