• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of ploscompComputational BiologyView this ArticleSubmit to PLoSGet E-mail AlertsContact UsPublic Library of Science (PLoS)
PLoS Comput Biol. Nov 2011; 7(11): e1002211.
Published online Nov 3, 2011. doi:  10.1371/journal.pcbi.1002211
PMCID: PMC3207943

Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons

Olaf Sporns, Editor

Abstract

The organization of computations in networks of spiking neurons in the brain is still largely unknown, in particular in view of the inherently stochastic features of their firing activity and the experimentally observed trial-to-trial variability of neural systems in the brain. In principle there exists a powerful computational framework for stochastic computations, probabilistic inference by sampling, which can explain a large number of macroscopic experimental data in neuroscience and cognitive science. But it has turned out to be surprisingly difficult to create a link between these abstract models for stochastic computations and more detailed models of the dynamics of networks of spiking neurons. Here we create such a link and show that under some conditions the stochastic firing activity of networks of spiking neurons can be interpreted as probabilistic inference via Markov chain Monte Carlo (MCMC) sampling. Since common methods for MCMC sampling in distributed systems, such as Gibbs sampling, are inconsistent with the dynamics of spiking neurons, we introduce a different approach based on non-reversible Markov chains that is able to reflect inherent temporal processes of spiking neuronal activity through a suitable choice of random variables. We propose a neural network model and show by a rigorous theoretical analysis that its neural activity implements MCMC sampling of a given distribution, both for the case of discrete and continuous time. This provides a step towards closing the gap between abstract functional models of cortical computation and more detailed models of networks of spiking neurons.

Author Summary

It is well-known that neurons communicate with short electric pulses, called action potentials or spikes. But how can spiking networks implement complex computations? Attempts to relate spiking network activity to results of deterministic computation steps, like the output bits of a processor in a digital computer, are conflicting with findings from cognitive science and neuroscience, the latter indicating the neural spike output in identical experiments changes from trial to trial, i.e., neurons are “unreliable”. Therefore, it has been recently proposed that neural activity should rather be regarded as samples from an underlying probability distribution over many variables which, e.g., represent a model of the external world incorporating prior knowledge, memories as well as sensory input. This hypothesis assumes that networks of stochastically spiking neurons are able to emulate powerful algorithms for reasoning in the face of uncertainty, i.e., to carry out probabilistic inference. In this work we propose a detailed neural network model that indeed fulfills these computational requirements and we relate the spiking dynamics of the network to concrete probabilistic computations. Our model suggests that neural systems are suitable to carry out probabilistic inference by using stochastic, rather than deterministic, computing elements.

Introduction

Attempts to understand the organization of computations in the brain from the perspective of traditional, mostly deterministic, models of computation, such as attractor neural networks or Turing machines, have run into problems: Experimental data suggests that neurons, synapses, and neural systems are inherently stochastic [1], especially in vivo, and therefore seem less suitable for implementing deterministic computations. This holds for ion channels of neurons [2], synaptic release [3], neural response to stimuli (trial-to-trial variability) [4], [5], and perception [6]. In fact, several experimental studies arrive at the conclusion that external stimuli only modulate the highly stochastic spontaneous firing activity of cortical networks of neurons [7], [8]. Furthermore, traditional models for neural computation have been challenged by the fact that typical sensory data from the environment is often noisy and ambiguous, hence requiring neural systems to take uncertainty about external inputs into account. Therefore many researchers have suggested that information processing in the brain carries out probabilistic, rather than logical, inference for making decisions and choosing actions [9][22]. Probabilistic inference has emerged in the 1960’s [23], as a principled mathematical framework for reasoning in the face of uncertainty with regard to observations, knowledge, and causal relationships, which is characteristic for real-world inference tasks. This framework has become tremendously successful in real-world applications of artificial intelligence and machine learning. A typical computation that needs to be carried out for probabilistic inference on a high-dimensional joint distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e001.jpg is the evaluation of the conditional distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e002.jpg (or marginals thereof) over some variables of interest, say An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e003.jpg, given variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e004.jpg. In the following, we will call the set of variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e005.jpg, which we condition on, the observed variables and denote it by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e006.jpg.

Numerous studies in different areas of neuroscience and cognitive science have suggested that probabilistic inference could explain a variety of computational processes taking place in neural systems (see [10], [11]). In models of perception the observed variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e007.jpg are interpreted as the sensory input to the central nervous system (or its early representation by the firing response of neurons, e.g., in the LGN in the case of vision), and the variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e008.jpg model the interpretation of the sensory input, e.g., the texture and position of objects in the case of vision, which might be encoded in the response of neurons in various higher cortical areas [15]. Furthermore, in models for motor control the observed variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e009.jpg often consist not only of sensory and proprioceptive inputs to the brain, but also of specific goals and constraints for a planned movement [24][26], whereas inference is carried out over the variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e010.jpg representing a motor plan or motor commands to muscles. Recent publications show that human reasoning and learning can also be cast into the form of probabilistic inference problems [27][29]. In these models learning of concepts, ranging from concrete to more abstract ones, is interpreted as inference in lower and successively higher levels of hierarchical probabilistic models, giving a consistent description of inductive learning within and across domains of knowledge.

In spite of this active research on the functional level of neural processing, it turned out to be surprisingly hard to relate the computational machinery required for probabilistic inference to experimental data on neurons, synapses, and neural systems. There are mainly two different approaches for implementing the computational machinery for probabilistic inference in “neural hardware”. The first class of approaches builds on deterministic methods for evaluating exactly or approximately the desired conditional and/or marginal distributions, whereas the second class relies on sampling from the probability distributions in question. Multiple models in the class of deterministic approaches implement algorithms from machine learning called message passing or belief propagation [30][33]. By clever reordering of sum and product operators occurring in the evaluation of the desired probabilities, the total number of computation steps are drastically reduced. The results of subcomputations are propagated as "messages" or "beliefs" that are sent to other parts of the computational network. Other deterministic approaches for representing distributions and performing inference are probabilistic population code (PPC) models [34]. Although deterministic approaches provide a theoretically sound hypothesis about how complex computations can possibly be embedded in neural networks and explain aspects of experimental data, it seems difficult (though not impossible) to conciliate them with other aspects of experimental evidence, such as stochasticity of spiking neurons, spontaneous firing, trial-to-trial variability, and perceptual multistability.

Therefore other researchers (e.g., [16][18], [35]) have proposed to model computations in neural systems as probabilistic inference based on a different class of algorithms, which requires stochastic, rather than deterministic, computational units. This approach, commonly referred to as sampling, focuses on drawing samples, i.e., concrete values for the random variables that are distributed according to the desired probability distribution. Sampling can naturally capture the effect of apparent stochasticity in neural responses and seems to be furthermore consistent with multiple experimental effects reported in cognitive science literature [17], [18]. On the conceptual side, it has proved to be difficult to implement learning in message passing and PPC network models. In contrast, following the lines of [36], the sampling approach might be well suited to incorporate learning.

Previous network models that implement sampling in neural networks are mostly based on a special sampling algorithm called Gibbs (or general Metropolis-Hastings) sampling [9], [17], [18], [37]. The dynamics that arise from this approach, the so-called Glauber dynamics, however are only superficially similar to spiking neural dynamics observed in experiments, rendering these models rather abstract. Building on and extending previous models, we propose here a family of network models, that can be shown to exactly sample from any arbitrary member of a well-defined class of probability distributions via their inherent network dynamics. These dynamics incorporate refractory effects and finite durations of postsynaptic potentials (PSPs), and are therefore more biologically realistic than existing approaches. Formally speaking, our model implements Markov chain Monte Carlo (MCMC) sampling in a spiking neural network. In contrast to prior approaches however, our model incorporates irreversible dynamics (i.e., no detailed balance) allowing for finite time PSPs and refractory mechanisms. Furthermore, we also present a continuous time version of our network model. The resulting stochastic dynamical system can be shown to sample from the correct distribution. In general, continuous time models arguably provide a higher amount of biological realism compared to discrete time models.

The paper is structured in the following way. First we provide a brief introduction to MCMC sampling. We then define the neural network model whose neural activity samples from a given class of probability distributions. The model will be first presented in discrete time together with some illustrative simulations. An extension of the model to networks of more detailed spiking neuron models which feature a relative refractory mechanism is presented. Furthermore, it is shown how the neural network model can also be formulated in continuous time. Finally, as a concrete simulation example we present a simple network model for perceptual multistability.

Results

Recapitulation of MCMC sampling

In machine learning, sampling is often considered the “gold standard” of inference methods, since, assuming that we can sample from the distribution in question, and assuming enough computational resources, any inference task can be carried out with arbitrary precision (in contrast to some deterministic approximate inference methods such as variational inference). However sampling from an arbitrary distribution can be a difficult problem in itself, as, e.g., many distributions can only be evaluated modulo a global constant (the partition function). In order to circumvent these problems, elaborate MCMC sampling techniques have been developed in machine learning and statistics [38]. MCMC algorithms are based on the following idea: instead of producing an ad-hoc sample, a process that is heuristically comparable to a global search over the whole state space of the random variables, MCMC methods produce a new sample via a “local search” around a point in the state space that is already (approximately) a sample from the distribution.

More formally, a Markov chain An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e011.jpg (in discrete time) is defined by a set An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e012.jpg of states (we consider for discrete time only the case where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e013.jpg has a finite size, denoted by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e014.jpg) together with a transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e015.jpg. The operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e016.jpg is a conditional probability distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e017.jpg over the next state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e018.jpg given a preceding state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e019.jpg. The Markov chain An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e020.jpg is started in some initial state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e021.jpg, and moves through a trajectory of states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e022.jpg via iterated application of the stochastic transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e023.jpg. More precisely, if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e024.jpg is the state at time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e025.jpg, then the next state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e026.jpg is drawn from the conditional probability distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e027.jpg. An important theorem from probability theory (see, e.g., p. 232 in [39]) states that if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e028.jpg is irreducible (i.e., any state in An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e029.jpg can be reached from any other state in An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e030.jpg in finitely many steps with probability An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e031.jpg) and aperiodic (i.e., its state transitions cannot be trapped in deterministic cycles), then the probability An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e032.jpg converges for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e033.jpg to a probability An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e034.jpg that does not depend on the initial state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e035.jpg. This state distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e036.jpg is called the invariant distribution of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e037.jpg. The irreducibility of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e038.jpg implies that it is the only distribution over the states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e039.jpg that is invariant under its transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e040.jpg, i.e.

equation image
(1)

Thus, in order to carry out probabilistic inference for a given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e042.jpg, it suffices to construct an irreducible and aperiodic Markov chain An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e043.jpg that leaves An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e044.jpg invariant, i.e., satisfies equation (1). Then one can answer numerous probabilistic inference questions regarding An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e045.jpg without any numerical computations of probabilities. Rather, one plugs in the observed values for some of the random variables (RVs) and simply collects samples from the conditional distribution over the other RVs of interest when the Markov chain approaches its invariant distribution.

A convenient and popular method for the construction of an operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e046.jpg for a given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e047.jpg is looking for operators An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e048.jpg that satisfy the following detailed balance condition,

equation image
(2)

for all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e050.jpg. A Markov chain that satisfies (2) is said to be reversible. In particular, the Gibbs and Metropolis-Hastings algorithms employ reversible Markov chains. A very useful property of (2) is that it implies the invariance property (1), and this is in fact the standard method for proving (1). However, as our approach makes use of irreversible Markov chains as explained below, we will have to prove (1) directly.

Neural sampling

Let An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e051.jpg be some arbitrary joint distribution over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e052.jpg binary variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e053.jpg that only takes on values An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e054.jpg. We will show that under a certain computability assumption on An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e055.jpg a network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e056.jpg consisting of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e057.jpg spiking neurons An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e058.jpg can sample from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e059.jpg using its inherent stochastic dynamics. More precisely, we show that the stochastic firing activity of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e060.jpg can be viewed as a non-reversible Markov chain that samples from the given probability distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e061.jpg. If a subset An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e062.jpg of the variables are observed, modelled as the corresponding neurons being “clamped” to the observed values, the remaining network samples from the conditional distribution of the remaining variables given the observables. Hence, this approach offers a quite natural implementation of probabilistic inference. It is similar to sampling approaches which have already been applied extensively, e.g., in Boltzmann machines, however our model is more biologically realistic as it incorporates aspects of the inherent temporal dynamics and spike-based communication of a network of spiking neurons. We call this approach neural sampling in the remainder of the paper.

In order to enable a network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e063.jpg of spiking neurons to sample from a distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e064.jpg of binary variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e065.jpg, one needs to specify how an assignment An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e066.jpg of values to these binary variables can be represented by the spiking activity of the network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e067.jpg and vice versa. A spike, or action potential, of a biological neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e068.jpg has a short duration of roughly An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e069.jpg. But the effect of such spike, both on the neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e070.jpg itself (in the form of refractory processes) and on the membrane potential of other neurons (in the form of postsynaptic potentials) lasts substantially longer, on the order of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e071.jpg to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e072.jpg. In order to capture this temporally extended effect of each spike, we fix some parameter An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e073.jpg that models the average duration of these temporally extended processes caused by a spike. We say that a binary vector An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e074.jpg is represented by the firing activity of the network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e075.jpg at time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e076.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e077.jpg iff:

equation image
(3)

In other words, any spike of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e079.jpg sets the value of the associated binary variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e080.jpg to 1 for a duration of length An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e081.jpg.

An obvious consequence of this definition is that the binary vector An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e082.jpg that is defined by the activity of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e083.jpg at time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e084.jpg does not fully capture the internal state of this stochastic system. Rather, one needs to take into account additional non-binary variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e085.jpg, where the value of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e086.jpg at time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e087.jpg specifies when within the time interval An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e088.jpg the neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e089.jpg has fired (if it has fired within this time interval, thereby causing An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e090.jpg at time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e091.jpg). The neural sampling process has the Markov property only with regard to these more informative auxiliary variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e092.jpg. Therefore our analysis of neural sampling will focus on the temporal evolution of these auxiliary variables. We adopt the convention that each spike of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e093.jpg sets the value of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e094.jpg to its maximal value An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e095.jpg, from which it linearly decays back to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e096.jpg during the subsequent time interval of length An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e097.jpg.

For the construction of the sampling network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e098.jpg, we assume that the membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e099.jpg of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e100.jpg at time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e101.jpg equals the log-odds of the corresponding variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e102.jpg to be active, and refer to this property as neural computability condition:

equation image
(4)

where we write An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e104.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e105.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e106.jpg for the current values An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e107.jpg of all other variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e108.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e109.jpg. Under the assumption we make in equation (4), i.e., that the neural membrane potential reflects the log-odds of the corresponding variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e110.jpg, it is required that each single neuron in the network can actually compute the right-hand side of equation (4), i.e., that it fulfills the neural computability condition.

A concrete class of probability distributions, that we will use as an example in the remainder, are Boltzmann distributions:

equation image
(5)

with arbitrary real valued parameters An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e112.jpg which satisfy An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e113.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e114.jpg (the constant An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e115.jpg ensures the normalization of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e116.jpg). For the Boltzmann distribution, condition (4) is satisfied by neurons An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e117.jpg with the standard membrane potential

equation image
(6)

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e119.jpg is the bias of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e120.jpg (which regulates its excitability), An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e121.jpg is the strength of the synaptic connection from neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e122.jpg to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e123.jpg, and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e124.jpg approximates the time course of the postsynaptic potential in neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e125.jpg caused by a firing of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e126.jpg with a constant signal of duration An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e127.jpg (i.e., a square pulse). As we will describe below, spikes of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e128.jpg are evoked stochastically depending on the current membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e129.jpg and the auxiliary variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e130.jpg.

The neural computability condition (4) links classes of probability distributions to neuron and synapse models in a network of spiking neurons. As shown above, Boltzmann distributions satisfy the condition if one considers point neuron models which compute a linear weighted sum of the presynaptic inputs. The class of distributions can be extended to include more complex distributions using a method proposed in [40] which is based on the following idea. Neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e131.jpg representing the variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e132.jpg is not directly influenced by the activities An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e133.jpg of the presynaptic neurons, but via intermediate nonlinear preprocessing elements. This preprocessing might be implemented by dendrites or other (inter-) neurons and is assumed to compute nonlinear combinations of the presynaptic activities An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e134.jpg (similar to a kernel). This allows the membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e135.jpg, and therefore the log-odds ratio on the right-hand side of (4), to represent a more complex function of the activities An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e136.jpg, giving rise to more complex joint distributions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e137.jpg. The concrete implementation of non-trivial directed and undirected graphical models with the help of preprocessing elements in the neural sampling framework is subject of current research. For the examples given in this study, we focus on the standard form of the membrane potential (6) of point neurons. As shown below, these spiking network models can emulate any Boltzmann machine (BM) [36].

A substantial amount of preceding studies has demonstrated that BMs are very powerful, and that the application of suitable learning algorithms for setting the weights An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e138.jpg makes it possible to learn and represent complex sensory processing tasks by such distributions [37], [41]. In applications in statistics and machine learning using such Boltzmann distributions, sampling is typically implemented by Gibbs sampling or more general reversible MCMC methods. However, it is difficult to model some neural processes, such as an absolute refractory period or a postsynaptic potential (PSP) of fixed duration, using a reversible Markov chain, but they are more conveniently modelled using an irreversible one. As we wish to keep the computational power of BMs and at the same time to augment the sampling procedure with aspects of neural dynamics (such as PSPs with fixed durations, refractory mechanisms) to increase biological realism, we focus in the following on irreversible MCMC methods (keeping in mind that this might not be the only possible way to achieve these goals).

Neural sampling in discrete time

Here we describe neural dynamics in discrete time with an absolute refractory period An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e139.jpg. We interpret one step of the Markov chain as a time step An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e140.jpg in biological real time. The dynamics of the variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e141.jpg, that describes the time course of the effect of a spike of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e142.jpg, are defined in the following way. An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e143.jpg is set to the value An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e144.jpg when neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e145.jpg fires, and decays by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e146.jpg at each subsequent discrete time step. The parameter An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e147.jpg is chosen to be some integer, so that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e148.jpg decays back to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e149.jpg in exactly An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e150.jpg time steps. The neuron can only spike (with a probability that is a function of its current membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e151.jpg) if its variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e152.jpg. If however, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e153.jpg, the neuron is considered refractory and it cannot spike, but its An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e154.jpg is reduced by 1 per time step. To show that these simple dynamics do indeed sample from the given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e155.jpg, we proceed in the following way. We define a joint distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e156.jpg which has the desired marginal distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e157.jpg. Further we formalize the dynamics informally described above as a transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e158.jpg operating on the state vector An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e159.jpg. Finally, in the Methods section, we show that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e160.jpg is the unique invariant distribution of this operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e161.jpg, i.e., that the dynamics described by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e162.jpg produce samples An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e163.jpg from the desired distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e164.jpg. We refer to sampling through networks with this stochastic spiking mechanism as neural sampling with absolute refractory period due to the persistent refractory process.

Given the distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e165.jpg that we want to sample from, we define the following joint distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e166.jpg over the neural variables:

equation image
(7)

This definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e168.jpg simply expresses that if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e169.jpg, then the auxiliary variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e170.jpg can assume any value in An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e171.jpg with equal probability. On the other hand An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e172.jpg necessarily assumes the value An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e173.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e174.jpg (i.e., when the neuron is in its resting state).

The state transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e175.jpg can be defined in a transparent manner as a composition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e176.jpg transition operators, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e177.jpg, where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e178.jpg only updates the variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e179.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e180.jpg of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e181.jpg, i.e., the neurons are updated sequentially in the same order (this severe restriction will become obsolete in the case of continuous time discussed below). We define the composition as An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e182.jpg, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e183.jpg is applied prior to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e184.jpg. The new values of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e185.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e186.jpg only depend on the previous value An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e187.jpg and on the current membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e188.jpg. The interesting dynamics take place in the variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e189.jpg. They are illustrated in Figure 1 where the arrows represent transition probabilities greater than 0.

Figure 1
Neuron model with absolute refractory mechanism.

If the neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e195.jpg is not refractory, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e196.jpg, it can spike (i.e., a transition from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e197.jpg to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e198.jpg) with probability

equation image
(8)

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e200.jpg is the standard sigmoidal activation function and the An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e201.jpg denotes the natural logarithm. The term An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e202.jpg is the current membrane potential, which depends on the current values of the variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e203.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e204.jpg. The term An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e205.jpg in (8) reflects the granularity of a chosen discrete time scale. If it is very fine (say one step equals one microsecond), then An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e206.jpg is large, and the firing probability at each specific discrete time step is therefore reduced. If the neuron in a state with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e207.jpg does not spike, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e208.jpg relaxes into the resting state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e209.jpg corresponding to a non-refractory neuron.

If the neuron is in a refractory state, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e210.jpg, its new variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e211.jpg assumes deterministically the next lower value An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e212.jpg, reflecting the inherent temporal process:

equation image
(9)

After the transition of the auxiliary variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e214.jpg, the binary variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e215.jpg is deterministically set to a consistent state, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e216.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e217.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e218.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e219.jpg.

It can be shown that each of these stochastic state transition operators An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e220.jpg leaves the given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e221.jpg invariant, i.e., satisfies equation (1). This implies that any composition or mixture of these operators An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e222.jpg also leaves An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e223.jpg invariant, see, e.g., [38]. In particular, the composition An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e224.jpg of these operators An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e225.jpg leaves An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e226.jpg invariant, which has a quite natural interpretation as firing dynamics of the spiking neural network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e227.jpg: At each discrete time step the variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e228.jpg are updated for all neurons An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e229.jpg, where the update of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e230.jpg takes preceding updates for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e231.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e232.jpg into account. Alternatively, one could also choose at each discrete time step a different order for updates according to [38]. The assumption of a well-regulated updating policy will be overcome in the continuous-time limit, i.e., in case where the neural dynamics are described as a Markov jump process. In the methods section we prove the following central theorem:

Theorem 1

An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e233.jpg is the unique invariant distribution of operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e234.jpg, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e235.jpg is aperiodic and irreducible and satisfies

equation image
(10)

The proof of this Theorem is provided by Lemmata 1 – 3 in the Methods section. The statement that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e237.jpg (which is composed of the operators An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e238.jpg) is irreducible and aperiodic ensures that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e239.jpg is the unique invariant distribution of the Markov chain defined by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e240.jpg, i.e., that irrespective of the initial network state the successive application of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e241.jpg explores the whole state space in a non-periodic manner.

This theorem guarantees that after a sufficient “burn-in” time (more precisely in the limit of an infinite “burn-in” time), the dynamics of the network, which are given by the transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e242.jpg, produce samples from the distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e243.jpg. As by construction An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e244.jpg, the Markov chain provides samples from the given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e245.jpg. Furthermore, the network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e246.jpg can carry out probabilistic inference for this distribution. For example, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e247.jpg can be used to sample from the posterior distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e248.jpg over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e249.jpg given An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e250.jpg. One just needs to clamp those neurons An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e251.jpg to the corresponding observed values. This could be implemented by injecting a strong positive (negative) current into the units with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e252.jpg (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e253.jpg). Then, as soon as the stochastic dynamics of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e254.jpg has converged to its invariant distribution, the averaged firing rate of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e255.jpg is proportional to the following desired marginal probability

equation image

In a biological neural system this result of probabilistic inference could for example be read out by an integrator neuron that counts spikes from this neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e257.jpg within a behaviorally relevant time window of a few hundred milliseconds, similarly as the experimentally reported integrator neurons in area LIP of monkey cortex [20], [21]. Another readout neuron that receives spike input from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e258.jpg could at the same time estimate An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e259.jpg for another RV An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e260.jpg. But valuable information for probabilistic inference is not only provided by firing rates or spike counts, but also by spike correlations of the neurons An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e261.jpg in An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e262.jpg. For example, the probability An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e263.jpg can be estimated by a readout neuron that responds to superpositions of EPSPs caused by near-coincident firing of neurons An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e264.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e265.jpg within a time interval of length An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e266.jpg. Thus, a large number of different probabilistic inferences can be carried out efficiently in parallel by readout neurons that receive spike input from different subsets of neurons in the network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e267.jpg.

Variation of the discrete time model with a relative refractory mechanism

For the previously described simple neuron model, the refractory process was assumed to last for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e268.jpg time steps, exactly as long as the postsynaptic potentials caused by each spike. In this section we relax this assumption by introducing a more complex and biologically more realistic neuron model, where the duration of the refractory process is decoupled from the duration An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e269.jpg of a postsynaptic potential. Thus, this model can for example also fire bursts of spikes with an interspike interval An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e270.jpg. The introduction of this more complex neuron model comes at the price that one can no longer prove that a network of such neurons samples from the desired distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e271.jpg. Nevertheless, if the sigmoidal activation function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e272.jpg is replaced by a different activation function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e273.jpg, one can still prove that the sampling is “locally correct”, as specified in equation (12) below. Furthermore, our computer simulations suggest that also globally the error introduced by the more complex neuron model is not functionally significant, i.e. that statistical dependencies between the RVs An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e274.jpg are still faithfully captured.

The neuron model with a relative refractory period is defined in the following way. Consider some arbitrary refractory function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e275.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e276.jpg, and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e277.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e278.jpg. The idea is that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e279.jpg models the readiness of the neuron to fire in its state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e280.jpg. This readiness has value An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e281.jpg when the neuron has fired at the preceding time step (i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e282.jpg), and assumes the resting state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e283.jpg when An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e284.jpg has dropped to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e285.jpg. In between, the readiness may take on any non-negative value according to the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e286.jpg. The function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e287.jpg does not need to be monotonic, allowing for example that it increases to high values in between, yielding a preferred interspike interval of a oscillatory neuron. The firing probability of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e288.jpg in state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e289.jpg is given by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e290.jpg, where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e291.jpg is an appropriate function of the membrane potential as described below. Thus this function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e292.jpg is closely related to the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e293.jpg (called afterpotential) in the spike response model [5] as well as to the self-excitation kernel in Generalized Linear Models [42]. In general, different neurons in the network may have different refractory profiles, which can be modeled by a different refractory function for each neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e294.jpg. However for the sake of notational simplicity we assume a single refractory function in the following.

In the presence of this refractory function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e295.jpg one needs to replace the sigmoidal activation function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e296.jpg by a suitable function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e297.jpg that satisfies the condition

equation image
(11)

for all real numbers An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e299.jpg. This equation can be derived (see Methods section Lemma 0) if one requires each neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e300.jpg to represent the correct distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e301.jpg over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e302.jpg conditioned the variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e303.jpg. One can show that, for any An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e304.jpg as above, there always exists a continuous, monotonic function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e305.jpg which satisfies this equation (see Lemma 0 in Methods). Unfortunately (11) cannot be solved analytically for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e306.jpg in general. Hence, for simulations we approximate the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e307.jpg for a given An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e308.jpg by numerically solving (11) on a grid and interpolating between the grid points with a constant function. Examples for several functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e309.jpg and the associated An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e310.jpg are shown in Figure 2B and Figure 2C respectively. Furthermore, spike trains emitted by single neurons with these refractory functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e311.jpg and the corresponding functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e312.jpg are shown in Figure 2D for the case of piecewise constant membrane potentials. This figure indicates, that functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e313.jpg that define a shorter refractory effect lead to higher firing rates and more irregular firing. It is worth noticing that the standard activation function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e314.jpg is the solution of equation (11) for the absolute refractory function, i.e., for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e315.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e316.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e317.jpg.

Figure 2
Neuron model with relative refractory mechanism.

The transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e333.jpg is defined for this model in a very similar way as before. However, for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e334.jpg, when the variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e335.jpg was deterministically reduced by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e336.jpg in the simpler model (yielding An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e337.jpg), this reduction occurs now only with probability An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e338.jpg. With probability An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e339.jpg the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e340.jpg sets An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e341.jpg, modeling the firing of another spike of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e342.jpg at this time point. The neural computability condition (4) remains unchanged, e.g., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e343.jpg for a Boltzmann distribution. A schema of the stochastic dynamics of this local state transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e344.jpg is shown in Figure 2A.

This transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e345.jpg has the following properties. In Lemma 0 in Methods it is proven that the unique invariant distribution of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e346.jpg, denoted as An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e347.jpg, gives rise to the correct marginal distribution over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e348.jpg, i.e.

equation image

This means that a neuron whose dynamics is described by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e350.jpg samples from the correct distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e351.jpg if it receives a static input from the other neurons in the network, i.e., as long as its membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e352.jpg is constant. Hence the “local” computation performed by such neuron can be considered as correct. If however, several neurons in the network change their states in a short interval of time, the joint distribution over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e353.jpg is in general not the desired one, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e354.jpg, where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e355.jpg denotes the invariant distribution of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e356.jpg. In the Methods section, we present simulation results that indicate that the error of the approximation to the desired Boltzmann distributions introduced by neural sampling with relative refractory mechanism is rather minute. It is shown that the neural sampling approximation error is orders of magnitudes below the one introduced by a fully factorized distribution (which amounts to assuming correct marginal distributions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e357.jpg and independent neurons).

To illustrate the sampling process with the relative refractory mechanism, we examine a network of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e358.jpg neurons. We aim to sample from a Boltzmann distribution (5) with parameters An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e359.jpg, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e360.jpg being randomly drawn from normal distributions. For the neuron model, we use the relative refractory mechanism shown in the mid row of Figure 2B. A detailed description of the simulation and the parameters used is given in the Methods section. A spike pattern of the resulting sampling network is shown in Figure 3A. The network features a sparse, irregular spike response with average firing rate of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e361.jpg. For one neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e362.jpg, indicated with orange spikes, the internal dynamics are shown in Figure 3B. After each action potential the neuron’s refractory function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e363.jpg drops to zero and reduces the probability of spiking again in a short time interval. The influence of the remaining network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e364.jpg is transmitted to neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e365.jpg via PSPs of duration An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e366.jpg and sums up to the fluctuating membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e367.jpg. As reflected in the highly variable membrane potential even this small network exhibits rich interactions. To represent the correct distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e368.jpg over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e369.jpg conditioned on An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e370.jpg, the neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e371.jpg continuously adapts its instantaneous firing rate. To quantify the precision with which the spiking network draws samples from the target distribution (5), Figure 3C shows the joint distribution of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e372.jpg neurons. For comparison we accompany the distribution of sampled network states with the result obtained from the standard Gibbs sampling algorithm (considered as the ground truth). Since the number of possible states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e373.jpg grows exponentially in the number of neurons, we restrict ourselves for visualization purposes to the distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e374.jpg of the gray shaded units and marginalize over the remaining network. The probabilities are estimated from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e375.jpg samples, i.e., from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e376.jpg successive states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e377.jpg of the Markov chain. Stochastic deviations of the estimated probabilities due to the finite number of samples are quite small (typical errors An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e378.jpg) and are comparable to systematic deviations due to the only locally correct computation of neurons with relative refractory mechanism. In the Methods section, we present further simulation results showing that the proposed networks consisting of neurons with relative refractory mechanism approximate the desired target distributions faithfully over a large range of distribution parameters.

Figure 3
Sampling from a Boltzmann distribution by spiking neurons with relative refractory mechanism.

In order to illustrate that the proposed sampling networks feature biologically quite realistic spiking dynamics, we present in the Methods section several neural firing statistics (e.g., the inter-spike interval histogram) of the network model. In general, the statistics computed from the model match experimentally observed statistics well. The proposed network models are based on the assumption of rectangular-shaped, renewal PSPs. More precisely, we define renewal (or non-additive) PSPs in the following way. Renewal PSPs evoked by a single synapse do not add up but are merely prolonged in their duration (according to equation (6)); renewal PSPs elicited at different synapses nevertheless add up in the normal way. In Methods we investigate the impact of replacing the theoretically ideal rectangular-shaped, renewal PSPs with biologically more realistic alpha-shaped, additive PSPs. Simulation results suggest that the network model with alpha-shaped PSPs does not capture the target distribution as accurately as with the theoretically ideal PSP shapes, statistical dependencies between the RVs An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e385.jpg are however still approximated reasonably well.

Neural sampling in continuous time

The neural sampling model proposed above was formulated in discrete time of step size An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e386.jpg, inspired by the discrete time nature of MCMC techniques in statistics and machine learning as well as to make simulations possible on digital computers. However, models in continuous time (e.g., ordinary differential equations) are arguably more natural and “realistic” descriptions of temporally varying biological processes. This gives rise to the question whether one can find a sensible limit of the discrete time model in the limit An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e387.jpg, yielding a sampling network model in continuous time. Another motivation for considering continuous time models for neural sampling is the fact that many mathematical models for recurrent networks are formulated in continuous time [5], and a comparison to these existing models would be facilitated. Here we propose a stochastically spiking neural network model in continuous time, whose states still represent correct samples from the desired probability distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e388.jpg at any time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e389.jpg. These types of models are usually referred to as Markov jump processes. It can be shown that discretizing this continuous time model yields the discrete time model defined earlier, which thus can be regarded as a version suitable for simulations on a digital computer.

We define the continuous time model in the following way. Let An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e390.jpg, for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e391.jpg, denote the firing times of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e392.jpg. The refractory process of this neuron, in analogy to Figure 1 and equation (8)-(9) for the case of discrete time, is described by the following differential equation for the auxiliary variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e393.jpg, which may now assume any nonnegative real number An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e394.jpg:

equation image
(12)

Here An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e396.jpg denotes Dirac’s Delta centered at the spike time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e397.jpg. This differential equation describes the following simple dynamics. The auxiliary variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e398.jpg decays linearly with time constant An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e399.jpg when the neuron is refractory, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e400.jpg. Once An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e401.jpg arrives at its resting state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e402.jpg it remains there, corresponding to the neuron being ready to spike again (more precisely, in order to avoid point measures we set it to a random value in An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e403.jpg, see Methods). In the resting state, the neuron has the probability density An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e404.jpg to fire at every time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e405.jpg. If it fires at An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e406.jpg, this results in setting An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e407.jpg, which is formalized in equation (12) by the sum of Dirac Delta’s An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e408.jpg. Here the current membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e409.jpg at time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e410.jpg is defined as in the discrete time case, e.g., by An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e411.jpg for the case of a Boltzmann distribution (5). The binary variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e412.jpg is defined to be 1 if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e413.jpg and 0 if the neuron is in the resting state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e414.jpg. Biologically, the term An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e415.jpg can again be interpreted as the value at time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e416.jpg of a rectangular-shaped PSP (with a duration of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e417.jpg) that neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e418.jpg evokes in neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e419.jpg. As the spikes are discrete events in continuous time, the probability of two or more neurons spiking at the same time is zero. This allows for updating all neurons in parallel using a differential equation.

In analogy to the discrete time case, the neural network in continuous time can be shown to sample from the desired distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e420.jpg, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e421.jpg is an invariant distribution of the network dynamics defined above. However, to establish this fact, one has to rely on a different mathematical framework. The probability distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e422.jpg of the auxiliary variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e423.jpg as a function of time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e424.jpg, which describes the evolution of the network, obeys a partial differential equation, the so-called Differential-Chapman-Kolmogorov equation (see [43]):

equation image
(13)

where the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e426.jpg, which captures the dynamics of the network, is implicitly defined by the differential equations (12) and the spiking probabilities. This operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e427.jpg is the continuous time equivalent to the transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e428.jpg in the discrete time case. The operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e429.jpg consists here of two components. The drift term captures the deterministic decay process of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e430.jpg, stemming from the term An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e431.jpg in equation (12). The jump term describes the non-continuous aspects of the path An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e432.jpg associated with “jumping” from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e433.jpg to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e434.jpg at the time An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e435.jpg when the neuron fires.

In the Methods section we prove that the resulting time invariant distribution, i.e., the distribution that solves An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e436.jpg, now denoted An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e437.jpg as it is not a function of time, gives rise to the desired marginal distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e438.jpg over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e439.jpg:

equation image
(14)

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e441.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e442.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e443.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e444.jpg otherwise. An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e445.jpg denotes Kronecker’s Delta with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e446.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e447.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e448.jpg otherwise. Thus, the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e449.jpg simply reflects the definition that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e450.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e451.jpg and 0 otherwise. For an explicit definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e452.jpg, a proof of the above statement, and some additional comments see the Methods section.

The neural samplers in discrete and continuous time are closely related. The model in discrete time provides an increasingly more precise description of the inherent spike dynamics when the duration An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e453.jpg of the discrete time step is reduced, causing an increase of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e454.jpg (such that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e455.jpg is constant) and therefore a reduced firing probability of each neuron at any discrete time step (see the term An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e456.jpg in equation (8)). In the limit of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e457.jpg approaching An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e458.jpg, the probability that two or more neurons will fire at the same time approaches An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e459.jpg, and the discrete time sampler becomes equal to the continuous time system defined above, which updates all units in parallel.

It is also possible to formulate a continuous time version of the neural sampler based on neuron models with relative refractory mechanisms. In the Methods section the resulting continuous time neuron model with a relative refractory mechanism is defined. Theoretical results similar to the discrete time case can be derived for this sampler (see Lemmata 9 and 10 in Methods): It is shown that each neuron “locally” performs the correct computation under the assumption of static input from the remaining neurons. However one can no longer prove in general that the global network samples from the target distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e460.jpg.

Demonstration of probabilistic inference with recurrent networks of spiking neurons in an application to perceptual multistability

In the following we present a network model for perceptual multistability based on the neural sampling framework introduced above. This simulation study is aimed at showing that the proposed network can indeed sample from a desired distribution and also perform inference, i.e., sample from the correct corresponding posterior distribution. It is not meant to be a highly realistic or exhaustive model of perceptual multistability nor of biologically plausible learning mechanisms. Such models would naturally require considerably more modelling work.

Perceptual multistability evoked by ambiguous sensory input, such as a 2D drawing (e.g., Necker cube) that allows for different consistent 3D interpretations, has become a frequently studied perceptual phenomenon. The most important finding is that the perceptual system of humans and nonhuman primates does not produce a superposition of different possible percepts of an ambiguous stimulus, but rather switches between different self-consistent global percepts in a spontaneous manner. Binocular rivalry, where different images are presented to the left and right eye, has become a standard experimental paradigm for studying this effect [44][47]. A typical pair of stimuli are the two images shown in Figure 4A. Here the percepts of humans and nonhuman primates switch (seemingly stochastically) between the two presented orientations. [16][18] propose that several aspects of experimental data on perceptual multistability can be explained if one assumes that percepts correspond to samples from the conditional distribution over interpretations (e.g., different 3D shapes) given the visual input (e.g., the 2D drawing). Furthermore, the experimentally observed fact that percepts tend to be stable on the time scale of seconds suggests that perception can be interpreted as probabilistic inference that is carried out by MCMC sampling which produces successively correlated samples. In [18] it is shown that this MCMC interpretation is also able to qualitatively reproduce the experimentally observed distribution of dominance durations, i.e., the distribution of time intervals between perceptual switches. However, in lack of an adequate model for sampling by a recurrent network of spiking neurons, theses studies could describe this approach only on a rather abstract level, and pointed out the open problem to relate this algorithmic approach to neural processes. We have demonstrated in a computer simulation that the previously described model for neural sampling could in principle fill this gap, providing a modelling framework that is on the one hand consistent with the dynamics of networks of spiking neurons, and which can on the other hand also be clearly understood from the perspective of probabilistic inference through MCMC sampling.

Figure 4
Modeling perceptual multistability as probabilistic inference with neural sampling.

In the following we model some essential aspects of an experimental setup for binocular rivalry with grating stimuli (see Figure 4A) in a recurrent network of spiking neurons with the previously described relative refractory mechanism. We assigned to each of the 217 neurons in the network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e468.jpg a tuning curve An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e469.jpg, centered around its preferred orientation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e470.jpg as shown in Figure 4B. The preferred orientations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e471.jpg of the neurons were chosen to cover the entire interval An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e472.jpg of possible orientations and were randomly assigned to the neurons. The neurons were arranged on a hexagonal grid as depicted in Figure 4F. Any two neurons with distance An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e473.jpg were synaptically connected (neighboring units had distance An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e474.jpg). We assume that these neurons represent neurons in the visual system that have roughly the same or neighboring receptive field, and that each neuron receives visual input from either the left or the right eye. The network connections were chosen such that neurons that have similar (very different) preferred orientations are connected with positive (negative) weights (for details see Methods section).

We examined the resulting distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e475.jpg over the An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e476.jpg dimensional network states. To provide an intuitive visualization of these high dimensional network states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e477.jpg, we resort to a 2-dimensional projection, the population vector of a state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e478.jpg (see Methods for details of the applied population vector decoding scheme). Only the endpoints of the population vectors are drawn (as colored points) in Figure 4D,E. The orientation of the population vector is assumed to correspond to the dominant orientation of the percept, and its distance from the origin encodes the strength of this percept. We also, somewhat informally, call the strength of a percept its coherence and a network state which represents a coherent percept a coherent network state. A coherent network state hence results in a population vector of large magnitude. Each direction of a population vector is color coded in Figure 4D,E, using the color code for directions shown on the right hand side of Figure 4F. In Figure 4D the distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e479.jpg of the network is illustrated by sampling of the network for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e480.jpg, with samples An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e481.jpg taken every millisecond. Each dot equals a sampled network state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e482.jpg. In a biological interpretation the spike response of the freely evolving network reflects spontaneous activity, since no observations, i.e., no external input, was added to the system. Figure 4D shows that the spontaneous activity of this simple network of spiking neurons moves preferably through coherent network states for all possible orientations due to the chosen recurrent network connections (being positive for neurons with similar preferred orientation and negative otherwise). This can directly be seen from the rare occurrence of population vectors with small magnitude (vectors close to the “center”) in Figure 4D.

To study percepts elicited by ambiguous stimuli, where inputs like in Figure 4A are shown simultaneously to the left and right eye during a binocular rivalry experiment, we provided ambiguous input to the network. Two cells with preferred orientation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e483.jpg and two cells with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e484.jpg were clamped to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e485.jpg. Additionally four neurons with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e486.jpg resp. An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e487.jpg were muted by clamping to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e488.jpg. This ambiguous input is incompatible with a coherent percept, as it corresponds to two orthogonal orientations presented at the same time. The resulting distribution over the state of the 209 remaining neurons is shown for a time span of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e489.jpg of simulated biological time (with samples taken every millisecond) in Figure 4E. One clearly sees that the network spends most of the time in network states that correspond to one of the two simultaneously presented input orientations (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e490.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e491.jpg), and virtually no time on orientations in between. This implements a sampling process from a bimodal conditional distribution. The black line marks a An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e492.jpg trace of network states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e493.jpg around a perceptual switch: The network remained in one mode of high probability – corresponding to one percept – for some period of time, and then quickly traversed the state space to another mode – corresponding to a different percept.

Three of the states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e494.jpg around this perceptual switch (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e495.jpg, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e496.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e497.jpg in Figure 4E) are explicitly shown in Figure 4F. Neurons An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e498.jpg that fired during the preceding interval of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e499.jpg ms (marked in gray in Figure 4G) are drawn in the respective color of their preferred orientation. Inactive neurons are drawn in white, and clamped neurons are marked by a black dot (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e500.jpg).

Figure 4G shows the action potentials of the An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e501.jpg non-clamped neurons during the same An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e502.jpg trace around the perceptual switch. One sees that the sampling process is expressed in this neural network model by a sparse, asynchronous and irregular spike response. It is worth mentioning that the average firing rate when sampling from the posterior distribution is only slightly higher than the average firing rate of spontaneous activity (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e503.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e504.jpg respectively), which is reminiscent of related experimental data [7]. Thus on the basis of the overall network activity it is indistinguishable whether the network carries out an inference task or freely samples from its prior distribution. It is furthermore notable, that a focus of the network activity on the two orientations that are given by the external input can be achieved in this model, in spite of the fact that only two of the An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e505.jpg neurons were clamped for each of them. This numerical relationship is reminiscent of standard data on the weak input from LGN to V1 that is provided in the brain [48], [49], and raises the question whether the proposed neural sampling model could provide a possible mechanism (under the modelling assumptions made above) for cortical processing of such numerically weak external inputs.

The distribution of the resulting dominance durations, i.e., the time between perceptual switches, for the previously described setup with ambiguous input is shown for a continuous run of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e506.jpg in Figure 4C (a similar method as in [18] was used to measure dominance durations, see Methods). This distribution can be approximated quite well by a Gamma distribution, which also provides a good fit to experimental data (see the discussion in [18]). We expect that also other features of the more abstract MCMC model for biological vision of [17], [18], such as contextual biases and traveling waves, will emerge in larger and more detailed implementations of the MCMC approach through the proposed neural sampling method in networks of spiking neurons.

Discussion

We have presented a spiking neural network that samples from a given probability distribution via its inherent network dynamics. In particular the network is able to carry out probabilistic inference through sampling. The model, based on assumptions about the underlying probability distribution (formalized by the neural computability condition) as well as on certain assumptions regarding the underlying MCMC model, provides one possible neural implementation of the “inference-by-sampling paradigm” emerging in computational neuroscience.

During inference the observations (i.e., the variables which we wish to condition on) are modeled in this study by clamping the corresponding neurons by strong external input to the observed binary value. Units which receive no input or input with vanishing contrast (stimulus intensity) are treated as unobserved. Using this admittedly quite simplistic model of the input, we observed in simulations that our network model exhibits the following property: The onset of a sensory stimulus reduces the variability of the firing activity, which represents (after stimulus onset) a conditional distribution, rather than the prior distribution (see the difference between panels D and E of Figure 5. It is tempting to compare these results to the experimental finding of reduced firing rate variability after stimulus onset observed in several cortical areas [50]. We wish to point out however, that a consistent treatment of zero contrast stimuli requires more thorough modelling efforts (e.g., by explicitly adding a random variable for the stimulus intensity [35], [51]), which is not the focus of the presented work.

Figure 5
Firing statistics of neural sampling networks.

Virtually all high-level computational tasks that a brain has to solve can be formalized as optimization problems, that take into account a (possibly large) number of soft or hard constraints. In typical applications of probabilistic inference in science and engineering (see e.g. [52], [53]) such constraints are encoded in e.g., conditional probability tables or factors. In a biological setup they could possibly be encoded through the synaptic weights of a recurrent network of spiking neurons. The solution of such optimizations problems in a probabilistic framework via sampling, as implemented in our model, provides an alternative to deterministic solutions, as traditionally implemented in neural networks (see, e.g., [54] for the case of constraint satisfaction problems). Whereas an attractor neural network converges to one (possibly approximate) solution of the problem, a stochastic network may alternate between different approximate solutions and stay the longest at those approximate solutions that provide the best fit. This might be advantageous, as given more time a stochastic network can explore more of the state space and avoid shallow local minima. Responses to ambiguous sensory stimuli [44][47] might be interpreted as an optimization with soft constraints. The interpretation of human thinking as sampling process solving an inference task, recently proposed in cognitive science [28], [55], [56], further emphasizes that considering neural activity as an inferential process via sampling promises to be a fruitful approach.

Our approach builds on, and extends, previous work where recurrent networks of non-spiking stochastic neurons (commonly considered in artificial neural networks) were shown to be able to carry out probabilistic inference through Gibbs sampling [36]. In [57] a first extension of this approach to a network of recurrently connected spiking neurons had been presented. The dynamics of the recurrently connected spiking neurons are described as stepwise sampling from the posterior of a temporal Restricted Boltzmann Machine (tRBM) by introducing a clever interpretation of the temporal spike code as time varying parameters of a multivariate Gaussian distribution. Drawing one sample from the posterior of a RBM is, by construction, a trivial one-step task. In contrast to our model, the model of [57] does not produce multiple samples from a fixed posterior distribution, given the fixed input, but produces exactly one sample consisting of the temporal sequence of the hidden nodes, given a temporal input sequence. Similar temporal models, sometimes called Bayesian filtering, also underlie the important contributions of [58] and [32]. In [32] every single neuron is described as hidden Markov Model (HMM) with two states. Instead of drawing samples from the instantaneous posterior distribution using stochastic spikes, [32] presents a deterministic spike generation with the intention to convey the analog probability value rather than discrete samples. The approach presented here can be interpreted as a biologically more realistic version of Gibbs sampling for a specific class of probability distributions by taking into account a spike-based communication, finite duration PSPs and refractory mechanisms. Other implementations based on different distributions (e.g., directed graphical models) and different sampling methods (e.g., reversible MCMC methods) are of course conceivable and worth exploring.

In a computer experiment (see Figure 4, we used our proposed network to model aspects of biological vision as probabilistic inference along the lines of argumentation put forward in [16][18]. Our model was chosen to be quite simplistic, just to demonstrate that a number of experimental data on the dynamics of spontaneous activity [51], [59], [60] and binocular rivalry [44][47] can in principle be captured by this approach. The main point of the modelling study is to show that rather realistic neural dynamics can support computational functions rigorously formalized as inference via sampling.

We have also presented a model of spiking dynamics in continuous time that performs sampling from a given probability distribution. Although computer simulations of biological networks of neurons often actually use discrete time, it is desirable to also have a sound approach for understanding and describing the network sampling dynamics in continuous time, as the latter is arguable a natural framework for describing temporal processes in biology. Furthermore comparison to many existing continuous time neuron and network models of neurons is facilitated.

We have made various simplifying assumption regarding neural processes, e.g., simple symbolic postsynaptic potentials in the form of step-functions (reminiscent of plateau potentials caused by dendritic NMDA spikes [61]). More accurate models for neurons have to integrate a multitude of time constants that represent different temporal processes on the physical, molecular, and genetic level. Hence the open problem arises, to which extent this multitude of time constants and other complex dynamics can be integrated into theoretical models of neural sampling. We have gone one first step in this direction by showing that in computer simulations the two temporal processes that we have considered (refractory processes and postsynaptic potentials) can approximately be decoupled. Furthermore, we have presented simulation results suggesting that more realistic alpha-shaped, additive EPSPs are compatible with the functionality of the proposed network model.

Finally, we want to point out that the prospect of using networks of spiking neurons for probabilistic inference via sampling suggests new applications for energy-efficient spike-based and massively parallel electronic hardware that is currently under development [62], [63].

Methods

We first provide details and proofs for the neural sampling models, followed by details for the computer simulations. Then we investigate typical firing statistics of individual neurons during neural sampling and examine the approximation quality of neural sampling with different neuron and synapse models.

Mathematical details

Notation

To keep the derivations in a compact form, we introduce the following notations. We define the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e511.jpg of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e512.jpg to be An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e513.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e514.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e515.jpg otherwise. Analogously we define An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e516.jpg. Let An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e517.jpg denote Kronecker’s Delta, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e518.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e519.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e520.jpg whereas An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e521.jpg denotes Dirac’s Delta, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e522.jpg. Furthermore An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e523.jpg is the indicator function of the set An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e524.jpg, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e525.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e526.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e527.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e528.jpg.

Details to neural sampling with absolute refractory period in discrete time

The following Lemmata 1 – 3 provide a proof of Theorem 1. For completeness we begin this paragraph with a recapitulation of the definitions stated in Results. We then identify some central properties of the joint probability distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e529.jpg and proof that the proposed network samples from the desired invariant distribution.

For a given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e530.jpg over the binary variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e531.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e532.jpg, the joint distribution over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e533.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e534.jpg is defined in the following way (see equation 7):

equation image

The assumption An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e536.jpg for all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e537.jpg is required to show the irreducibility of the Markov chain, a prerequisite to ensure the uniqueness of the invariant distribution of the MCMC dynamics. Furthermore, for the given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e538.jpg we define the functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e539.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e540.jpg which map An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e541.jpg:

equation image

Instead of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e543.jpg we simply write An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e544.jpg in the following.

Lemma 1. The distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e545.jpg has conditional distributions of the following form:

equation image

These results can also be written more compactly in the following form: An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e547.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e548.jpg.

Proof. Here we use the fact that the logistic function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e549.jpg is the inverse of the logit function, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e550.jpg.

equation image

This also shows that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e552.jpg is independent from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e553.jpg given An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e554.jpg, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e555.jpg. Now we show the second relation using Bayes’ rule:

equation image

In order to facilitate the verification of the next two Lemmata, we first restate the definition of the operators An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e557.jpg in a more concise way:

equation image

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e559.jpg.

Lemma 2. For all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e560.jpg the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e561.jpg leaves the conditional distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e562.jpg invariant.

Proof. For sake of simplicity, denote An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e563.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e564.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e565.jpg. We have to show An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e566.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e567.jpg.

First we show An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e568.jpg using An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e569.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e570.jpg (which results from Lemma 1):

equation image

Here we used the definition of the logistic function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e572.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e573.jpg.

Now we show An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e574.jpg:

equation image

Here we used An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e576.jpg.

It is trivial to show An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e577.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e578.jpg as An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e579.jpg. Here we used the facts that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e580.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e581.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e582.jpg by definition.

Lemma 3. For all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e583.jpg the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e584.jpg leaves the distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e585.jpg invariant.

Proof. We start from Lemma 2, which states that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e586.jpg leaves the conditional distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e587.jpg invariant:

equation image

Here we used the relations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e589.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e590.jpg as well as An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e591.jpg which directly follow from the definitions of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e592.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e593.jpg.

Finally, we can verify that the composed operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e594.jpg samples from the given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e595.jpg.

Theorem 1. An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e596.jpg is the unique invariant distribution of operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e597.jpg.

Proof. As all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e598.jpg leave An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e599.jpg invariant, so does the concatenation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e600.jpg. To ensure that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e601.jpg is the unique invariant distribution, we have to show that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e602.jpg is irreducible and aperiodic. An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e603.jpg is aperiodic as the transition probabilities An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e604.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e605.jpg (this follows from the assumption An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e606.jpg made above).

The operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e607.jpg is also irreducible for the following reason. First we see that from any state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e608.jpg in at most An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e609.jpg steps we can get to the zero-state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e610.jpg (and stay there) with non-zero probability, as An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e611.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e612.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e613.jpg. Furthermore, it can be seen that any state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e614.jpg can be reached from the zero-state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e615.jpg in at most An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e616.jpg steps since An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e617.jpg for any value of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e618.jpg. Hence every final state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e619.jpg can be reached from every starting state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e620.jpg in at most An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e621.jpg steps with non-vanishing probability.

Details to neural sampling with a relative refractory period in discrete time

We augment the neuron model with a relative refractory period described by a function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e622.jpg. We first ensure existence of the corresponding function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e623.jpg. Based on these functions we then introduce the transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e624.jpg of the Markov chain. This operator is shown to entail correct “local” computations.

Lemma 4. Let An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e625.jpg be a tuple of non-negative real numbers, with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e626.jpg and at least one element An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e627.jpg. This defines the refractory function via An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e628.jpg. There exists a unique An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e629.jpg function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e630.jpg with the following property An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e631.jpg:

equation image
(15)

Furthermore, the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e633.jpg has the property:

equation image

Proof. Let An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e635.jpg; we know that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e636.jpg. We define the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e637.jpg:

equation image

We can see that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e639.jpg is a positive An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e640.jpg function on An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e641.jpg. Furthermore, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e642.jpg is defined as a sum of functions of the form An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e643.jpg. Each factor An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e644.jpg is positive and strictly monotonous. Therefore, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e645.jpg is strictly monotonous on An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e646.jpg with the limits:

equation image

Hence the equation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e648.jpg has a unique solution for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e649.jpg called An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e650.jpg for all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e651.jpg. From applying the implicit function theorem to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e652.jpg it follows that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e653.jpg is An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e654.jpg.

From here on, with the letter An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e655.jpg we will denote the function characterized by the above Lemma for the given tuple An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e656.jpg (which denotes the chosen refractory function).

Definition 1. Define An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e657.jpg. The transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e658.jpg is defined in the following way for all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e659.jpg:

equation image

with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e661.jpg.

Lemma 5. For all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e662.jpg the unique invariant distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e663.jpg of the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e664.jpg fulfills An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e665.jpg. This means, for a constant configuration An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e666.jpg, the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e667.jpg produces samples An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e668.jpg from the correct conditional distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e669.jpg.

Proof. We define:

equation image

where the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e671.jpg is defined as:

equation image

It is trivial to see that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e673.jpg has the correct marginal distribution over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e674.jpg:

equation image

We now show that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e676.jpg is the unique invariant distribution of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e677.jpg. Because of the definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e678.jpg, we only have to show that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e679.jpg is the unique invariant distribution of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e680.jpg. We denote An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e681.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e682.jpg, i.e., we have to show An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e683.jpg.

It is trivial to show An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e684.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e685.jpg, as there is only one non-vanishing element of transition operator, namely An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e686.jpg:

equation image

Here we used An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e688.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e689.jpg and the definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e690.jpg.

Now we show An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e691.jpg starting from equation (15) and additionally using the relations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e692.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e693.jpg as well as the definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e694.jpg. We define for the sake of simplicity An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e695.jpg:

equation image

We finally show An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e697.jpg, using the definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e698.jpg:

equation image

The argument that the transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e700.jpg is aperiodic and irreducible is similar to the one presented in Lemma 1.

Details to neural sampling with an absolute refractory period in continuous time

In contrast to the discrete time model we define the state space of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e701.jpg to be An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e702.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e703.jpg, i.e., as the union of the positive real numbers and a small interval An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e704.jpg. We will define the sampling operator in such a way that after neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e705.jpg was refractory for exactly its refractory period An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e706.jpg, its refractory variable An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e707.jpg is uniformly placed in the small interval An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e708.jpg, which represents now the resting state and replaces An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e709.jpg. This avoids point measures (Dirac’s Delta) on the value An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e710.jpg. This system is still exactly equivalent to the system discussed in the main paper, as all spike-transition probabilities of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e711.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e712.jpg are constant. Hence, it does not matter which values An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e713.jpg assumes with respect to the spike mechanism during its non-refractory period as long as An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e714.jpg.

Definition 2. For a given distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e715.jpg over the binary variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e716.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e717.jpg, we define a joint distribution over An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e718.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e719.jpg in the following way:

equation image

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e721.jpg is the refractory resting state interval. In accordance with this definition we can also write An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e722.jpg.

Lemma 6. The distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e723.jpg has the following marginal distribution:

equation image

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e725.jpg.

Definition 3. For An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e726.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e727.jpg the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e728.jpg is defined in the following way for a function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e729.jpg:

equation image

where the functional An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e731.jpg is defined as the one-sided limit from above at 0:

equation image

The operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e733.jpg is defined in the following way for a probability distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e734.jpg on An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e735.jpg:

equation image

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e737.jpg denotes the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e738.jpg of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e739.jpg where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e740.jpg is held constant and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e741.jpg.

The transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e742.jpg defines the following Fokker-Planck equation for a time-dependent distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e743.jpg:

equation image

The jump and drift functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e745.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e746.jpg associated to the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e747.jpg are given by:

equation image

Lemma 7. The operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e749.jpg leaves the conditional distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e750.jpg invariant with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e751.jpg, i.e.:

equation image

Proof. This is easy to proof using calculus and the relations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e753.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e754.jpg.

Lemma 8. An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e755.jpg is an invariant distribution of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e756.jpg, i.e., it is a solution to the invariant Fokker-Planck equation:

equation image

Proof. We observe that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e758.jpg for a constant An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e759.jpg (which is not a function of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e760.jpg). Hence:

equation image

The Lemma follows then from the definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e762.jpg.

Details to neural sampling with a relative refractory period in continuous time

As already assumed in the case of the absolute refractory sampler in continuous time, we define the state space of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e763.jpg to be An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e764.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e765.jpg.

Lemma 9. Let An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e766.jpg be a continuous, non-negative function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e767.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e768.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e769.jpg. There exists a unique An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e770.jpg function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e771.jpg with the following property An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e772.jpg:

equation image
(16)

Proof. We define the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e774.jpg in the following way:

equation image

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e776.jpg. From An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e777.jpg we can follow that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e778.jpg is non-negative. An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e779.jpg is differentiable with the derivative:

equation image

Hence An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e781.jpg is strictly monotonously increasing. Furthermore, the following relations hold:

equation image

Therefore the equation:

equation image

has exactly one solution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e784.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e785.jpg in An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e786.jpg. From applying the implicit function theorem to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e787.jpg it follows that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e788.jpg is An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e789.jpg.

Definition 4. For all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e790.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e791.jpg the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e792.jpg is defined in the following way for a function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e793.jpg:

equation image

The transition operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e795.jpg defines the following Fokker-Planck equation for a time-dependent distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e796.jpg:

equation image

The jump and drift functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e798.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e799.jpg associated to the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e800.jpg are given by:

equation image

Lemma 10. For all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e802.jpg the invariant distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e803.jpg of the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e804.jpg fulfills An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e805.jpg.

Proof. We define the distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e806.jpg as:

equation image

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e808.jpg. By applying the operator An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e809.jpg to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e810.jpg one can verify that An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e811.jpg holds using the definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e812.jpg given in (16). Furthermore we can compute the ratio:

equation image

Details to the computer simulations

The simulation results shown in Figure 2, Figure 3 and Figure 4 used the biologically more realistic neuron model with the relative refractory mechanism. During all experiments the first second of simulated time was discarded as burn-in time. The full list of parameters defining the experimental setup is given in Table 1. All occurring joint probability distributions are Boltzmann distributions of the form given in equation (5). Example Python [64] scripts for neural sampling from Boltzmann distributions are available on request and will be provided on our webpage. The example code comprises networks with both absolute and relative refractory mechanism. It requires standard Python packages only and is readily executable.

Table 1
List of parameters of the computer simulations.

Details to Figure 2: Neuron model with relative refractory mechanism

The three refractory functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e866.jpg of panel (B) as well as all other simulation parameters are listed in Table 1. Panel (C) shows the corresponding functions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e867.jpg, which result from numerically solving equation (11). The spike patterns in panel (D) show the response of the neurons when the membrane potential is low (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e868.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e869.jpg) or high (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e870.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e871.jpg). These membrane potentials encode An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e872.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e873.jpg, respectively according to (3) and (4). The binary state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e874.jpg is indicated by gray shaded areas of duration An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e875.jpg after each spike.

Details to Figure 3: Sampling from a Boltzmann distribution by spiking neurons with relative refractory mechanism

We examined the spike response of a network of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e876.jpg randomly connected neurons which sampled from a Boltzmann distribution. The excitabilities An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e877.jpg as well as the synaptic weights An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e878.jpg were drawn from Gaussian distributions (with diagonal elements An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e879.jpg). For the full list of parameters please refer to Table 1. One second of the arising spike pattern is shown in panel (A). The average firing rate of the network was An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e880.jpg. To highlight the internal dynamics of the neuron model, the values of the refractory function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e881.jpg, the membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e882.jpg and the instantaneous firing rate An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e883.jpg of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e884.jpg (indicated with red spikes) are shown in panel (B). Here, the instantaneous firing rate An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e885.jpg is defined for the discrete time Markov chain as

equation image
(17)

As stated before, the neuron model with relative refractory mechanism An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e887.jpg does not entail the correct overall invariant distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e888.jpg. To estimate the impact of this approximation on the joint network dynamics, we compared the distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e889.jpg over five neurons (indicated by gray background in A) in the spiking network with the correct distribution obtained from Gibbs sampling. The probabilities were estimated from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e890.jpg samples. A more quantitative analysis of the approximation quality of neural sampling with a relative refractory mechanism is provided below.

Details to Figure 4: Modeling perceptual multistability as probabilistic inference with neural sampling

We demonstrate probabilistic inference and learning in a network of orientation selective neurons. As a simple model we consider a network of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e891.jpg neurons on a hexagonal grid as shown in panel (F). Any two neurons with distance An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e892.jpg were synaptically connected (neighboring units had distance An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e893.jpg). For the remaining parameters of the network and neuron model please refer to Table 1. Each neuron featured a An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e894.jpg-periodic tuning curve as depicted in panel (B):

equation image
(18)

with base sensitivity An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e896.jpg, contrast An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e897.jpg, peakedness An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e898.jpg and preferred orientation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e899.jpg. The preferred orientations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e900.jpg of the neurons were chosen to cover the entire interval An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e901.jpg of possible orientations with equal spacing and were randomly assigned to the neurons.

For simplicity we did not incorporate the input dynamics in our probabilistic model, but rather trained the network directly like a fully visible Boltzmann machine. We used for this purpose a standard Boltzmann machine learning rule known as contrastive divergence [41], [65]. This learning rule requires posterior samples An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e902.jpg, i.e., network states under the influence of the present input, and approximate prior samples An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e903.jpg, which reflect the probability distribution of the network in the absence of stimuli. The update rules for synaptic weights and neuronal excitabilities read:

equation image
(19)

While more elaborate policies can speed up convergence, we simply used a global learning rate An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e905.jpg which was constant in time. The values of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e906.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e907.jpg were initialized at An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e908.jpg. We generated binary training patterns in the following way:

  1. A global orientation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e909.jpg was drawn uniformly from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e910.jpg,
  2. each neuron was independently set to be active with probability An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e911.jpg,
  3. the resulting network state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e912.jpg was taken as posterior sample.

To obtain an approximate prior sample An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e913.jpg we let the network run for a short time freely starting from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e914.jpg. The variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e915.jpg were also assumed to be observed with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e916.jpg iid. uniformly in An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e917.jpg if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e918.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e919.jpg otherwise. After evolving freely for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e920.jpg time steps, the resulting network state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e921.jpg was taken as approximate prior sample and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e922.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e923.jpg were updated according to (19). This process was repeated An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e924.jpg times. As a result, neurons with similar preferred orientations featured excitatory synaptic connections (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e925.jpg  = mean An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e926.jpg standard deviation of weight distribution), those with dissimilar orientations maintained inhibitory synapses (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e927.jpg). Here, preferred orientations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e928.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e929.jpg are defined as similar if An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e930.jpg, otherwise they are dissimilar. Neuronal biases converged to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e931.jpg.

We illustrate the learned prior distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e932.jpg of the network through sampled states when the network evolved freely. As seen in panel (D), the population vector – a 2-dimensional projection of the high dimensional network state – typically reflected an arbitrary, yet coherent, orientation (for the definition of the population vector see below). Each dot represents a sampled network state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e933.jpg.

To apply an ambiguous cue, we clamped An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e934.jpg out of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e935.jpg neurons: Two units with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e936.jpg and two with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e937.jpg were set active, two units with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e938.jpg and two with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e939.jpg were set inactive. This led to a bimodal posterior distribution as shown in panel (E). The sampling network represented this distribution by encoding either global perception separately: The trace of network states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e940.jpg roamed in one mode for multiple steps before quickly crossing the state space towards the opposite percept.

We define the population vector An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e941.jpg of a network state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e942.jpg as a function of the preferred orientations of all active units:

equation image
(20)

This definition of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e944.jpg is not based on the preferred orientations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e945.jpg which are used for generating external input to the network from a given stimulus with orientation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e946.jpg. It is rather based on the preferred orientations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e947.jpg measured from the network response. We used population vector decoding based on the measured values An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e948.jpg, as they are conceptually closer to experimentally measurable preferred orientations, and this decoding hence does not require knowledge of the (unobservable) An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e949.jpg. For every neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e950.jpg the preferred orientation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e951.jpg was measured in the following way. We estimated a tuning curve An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e952.jpg by a van-Mises fit (of the form (18)) to data from stimulation trials in which neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e953.jpg was not clamped, i.e., where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e954.jpg was only stimulated by recurrent input (feedforward input was modeled by clamping 8 out of 217 neurons as a function of stimulus orientation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e955.jpg as before). Due to the structured recurrent weights, the experimentally measured tuning curves An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e956.jpg were found to be reasonably close to the tuning curves An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e957.jpg used for external stimulation. An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e958.jpg was set to the preferred orientation of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e959.jpg (localization parameter of the van-Mises fit). The measured values An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e960.jpg turned out to be consistent with the preferred orientations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e961.jpg (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e962.jpg averaged over all An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e963.jpg neurons). The mean and standard deviation of the remaining parameter values An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e964.jpg, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e965.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e966.jpg of the fitted tuning curves An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e967.jpg are listed in Table 1 next to the ones used for stimulation.

The population vector An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e968.jpg was defined in (20) with the argument An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e969.jpg (instead of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e970.jpg) as orthogonal orientations should cancel each other and neighborhood relations should be respected. For example neurons with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e971.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e972.jpg contribute similarly to the population vector for small An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e973.jpg. But counter to intuition the population vector of a state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e974.jpg with dominant orientation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e975.jpg will point into direction An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e976.jpg. For visualization in panel (D) and (E) we therefore rescaled the population vector: If An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e977.jpg in polar coordinates, then the dot is located at An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e978.jpg in accord with intuition. The black semicircles equal An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e979.jpg.

The population vector An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e980.jpg was also used for measuring the dominance durations shown in panel (C). To this An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e981.jpg was divided into An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e982.jpg areas: (a) An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e983.jpg, (b) An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e984.jpg, (c) An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e985.jpg. We detected a perceptual switch when the network state entered area (a) or (c) while the previous perception was (c) or (a), respectively.

In panel (F) neurons An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e986.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e987.jpg are plotted with their preferred orientation color code, inactive neurons are displayed in white. Cells marked by a dot (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e988.jpg) were part of the observed variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e989.jpg. The three network states correspond to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e990.jpg with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e991.jpg, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e992.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e993.jpg in the spike pattern in panel (G). The spike pattern shows the response of the freely evolving units around a perceptual switch during sampling from the posterior distribution. The corresponding trace of the population vector is drawn as black line in panel (E). The width of the light-gray shaded areas in the spike pattern equals the PSP duration An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e994.jpg, i.e., neurons that spiked in these intervals were active in the corresponding state in (F).

Firing statistics of neural sampling networks

In previous sections it was shown that a spiking neural network can draw samples from a given joint distribution which is in a well-defined class of probability distributions (see the neural computability condition (4)). Here, we examine some statistics of individual neurons in a sampling network which are commonly used to analyze experimental data from recordings. The spike trains and membrane potential data are taken from the simulation presented in Figure 3.

Figure 5A,B exemplarily show the distribution of the membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e995.jpg and the interspike interval (ISI) histogram of a single neuron, namely neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e996.jpg which was already considered in Figure 3B. The responses of other neurons yield qualitatively similar statistics. The bell-shaped distribution of the membrane potential is commonly observed in neurons embedded in an active network [66]. The ISI histogram reflects the reduced spiking probability immediately after an action potential due the refractory mechanism. Interspike intervals larger than the refractory time constant An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e997.jpg roughly follow an exponential distribution. Similar ISI distributions were observed during in-vivo recordings in awake, behaving monkeys [67].

Figure 5C shows a scatterplot of the coefficient of variation (CV) of the ISIs versus the average ISI for each neuron in the network. The neurons exhibited a variety of average firing rates between An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e998.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e999.jpg. Most of the neurons responded in a highly irregular manner with a CV An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1000.jpg. Neurons with high firing rates had a slightly lower CV due to the increased influence of the refractory mechanism The dashed line marks the CV of a Poisson process, i.e., a memoryless spiking behavior. The CV of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1001.jpg is marked by a cross. The structure of this plot resembles, e.g., data from recordings in behaving macaque monkeys [68] (but note the lower average firing rate).

Approximation quality of neural sampling with different neuron and synapse models

The theory of the neuron model with absolute refractory mechanism guarantees sampling form the correct distribution. In contrast, the theory for the neuron model with a relative refractory mechanism only shows that the sampling process is “locally correct”, i.e., that it would yield correct conditional distributions An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1002.jpg for each individual neuron if the state of the remaining network An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1003.jpg stayed constant. Therefore, the stationary distribution of the sampling process with relative refractory mechanism only provides an approximation to the target distribution. In the following we examine the approximation quality and robustness of sampling networks with different refractory mechanisms for target Boltzmann distributions with parameters randomly drawn from different distributions. Furthermore, we investigate the effect of additive PSP shapes with more realistic time courses.

We generated target Boltzmann distributions with randomly drawn weights An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1004.jpg and biases (excitabilities) An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1005.jpg and computed the similarity between these reference distributions and the corresponding neural sampling approximations. The setup of these simulations is the same as for the simulation presented in Figure 3. As we aimed to compare the distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1006.jpg sampled by the network with the exact Boltzmann distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1007.jpg, we reduced the number of neurons per network to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1008.jpg. This resulted in a state space of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1009.jpg possible network states An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1010.jpg for which the normalization constant for the target Boltzmann distribution could be computed exactly. The weight matrix An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1011.jpg was constraint to be symmetric with vanishing diagonal. Off-diagonal elements were drawn from zero-mean normal distributions with three different standard deviations An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1012.jpg, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1013.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1014.jpg, whereas the An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1015.jpg were sampled from the same distribution as in Figure 3. For every value of the hyperparameter An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1016.jpg we generated 100 random distributions. For Boltzmann distributions with small weights (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1017.jpg), the RVs are nearly independent, whereas distributions with intermediate weights (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1018.jpg) show substantial statistical dependencies between RVs. For very large weights (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1019.jpg), the probability mass of the distributions is concentrated on very few states (usually 90% on less than 10 out of the An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1020.jpg states). Hence, the range of the hyperparameter An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1021.jpg considered here covers a range a very different distributions.

The approximation quality of the sampled distribution was measured in terms of the Kullback-Leibler divergence between the target distribution An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1022.jpg and the neural approximation An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1023.jpg

equation image
(21)

We estimated An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1025.jpg from An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1026.jpg samples for each simulation trial using a Laplace estimator, i.e., we added a priori An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1027.jpg to the number of occurrences of each state An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1028.jpg.

Table 2 shows the means and the standard deviations of the Kullback-Leibler divergences between the target Boltzmann distributions and the estimated approximations stemming from neural sampling networks with three different neuron and synapse models: the exact model with absolute refractory mechanism and two models with different relative refractory mechanisms shown in the bottom and middle row in Figure 2B. Additionally, as a reference, we provide the (analytically calculated) Kullback-Leibler divergences for fully factorized distributions, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1029.jpg with correct marginals An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1030.jpg but independent variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1031.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1032.jpg.

Table 2
Approximation quality of networks with different refractory mechanisms.

The absolute refractory model provides the best results as we expected due to the theoretical guarantee to sample from the correct distribution (the non-zero Kullback-Leibler divergence is caused by the estimation from a finite number of samples). The models with relative refractory mechanism provide faithful approximations for all values of the hyperparameter An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1054.jpg considered here. These relative refractory models are characterized by the theory to be “locally correct” and turn out to be much more accurate approximations than fully factorized distributions if substantial statistical dependencies between the RVs are present (i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1055.jpg, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1056.jpg). As expected, a late recovery of the refractory function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1057.jpg is beneficial for the approximation quality of the model as it is closer to an absolute refractory mechanism. Figure 6 shows the full histograms of the Kullback-Leibler divergences for the intermediate weights group (An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1058.jpg). Systematic deviations due to the relative refractory mechanism are on the same order as the effect of estimating from finite samples (as can be seen, e.g., from a comparison with the absolute refractory model which has 0 systematic error). For completeness, we mention that the divergences of the fully factorized distributions of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1059.jpg out of the An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1060.jpg networks with An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1061.jpg are not shown in the plot.

Figure 6
Comparison of neural sampling with different neuron and synapse models.

The theorems presented in this article assumed renewed (i.e., non-additive), rectangular PSPs. In the following we examine the effect of additive PSPs with more realistic time courses. We define additive, alpha-shaped PSPs in the following way. The influence An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1063.jpg of each presynaptic neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1064.jpg on the postsynaptic membrane potential An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1065.jpg is modeled by convolving the input spikes with a kernel An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1066.jpg:

equation image
(22)

where An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1068.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1069.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1070.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1071.jpg, and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1072.jpg for An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1073.jpg are the spike times of the presynaptic neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1074.jpg. The time constant governing the rising edge of the PSPs was set to An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1075.jpg. The time constant controlling the falling edge was chosen equal to the duration of rectangular PSPs, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1076.jpg. The scaling parameter An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1077.jpg was set such that the time integral over a single PSP matches the time integral over the theoretically optimal rectangular PSP, i.e., An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1078.jpg. These parameters display a simple and reasonable choice for the purpose of this study (an optimization of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1079.jpg, An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1080.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1081.jpg is likely to yield an improved approximation quality). Figure 7A shows the resulting shape of the non-rectangular PSP. Furthermore the time course of the function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1082.jpg caused by a single spike of neuron An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1083.jpg is shown in order to illustrate that the time constants of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1084.jpg and of a PSP are closely related due to the assumption An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1085.jpg made above. Preliminary and non-exhaustive simulations seem to suggest that the choice An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1086.jpg yields better approximation quality than setting An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1087.jpg or An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1088.jpg; however it is very well possible that a mismatch between An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1089.jpg and An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1090.jpg can be compensated for by adapting other parameters, e.g., the PSP magnitude or a specific choice of the refractory function An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1091.jpg. Figure 7B shows the results of an experiment, similar to the one presented in Figure 3C , with additive, alpha-shaped PSPs and relative refractory mechanism. While differences to Gibbs sampling results are visible, the spiking network still captures dependencies between the binary random variables quite well.

Figure 7
Sampling from a Boltzmann distribution with more realistic PSP shapes.

For a quantitative analysis of the approximation quality, we repeated the experiment of Figure 6 with additive, alpha-shaped PSPs (shown as green bars). The Kullback-Leibler divergence An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1101.jpg to the true distribution is clearly higher compared to the case of renewed, rectangular PSPs. Still networks with this more realistic synapse model account for dependencies between the random variables An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1102.jpg and yield a better approximation of An external file that holds a picture, illustration, etc.
Object name is pcbi.1002211.e1103.jpg than fully factorized distributions.

Acknowledgments

We would like to thank Mihai Petrovici, Robert Legenstein and Samuel Gershman for helpful discussions.

Footnotes

The authors have declared that no competing interests exist.

This paper was written under partial support by the European Union project #FP7-237955 (FACETS-ITN), project #FP7-269921 (BrainScaleS), project #FP7-216593 (SECO), project #FP7-506778 (PASCAL2) and project #FP7-243914 (BRAIN-I-NETS). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1. Rolls ET, Deco G. Oxford University Press; 2010. The Noisy Brain: Stochastic Dynamics as a Principle of Brain Function. [PubMed]
2. Cannon R, O’Donnell C, Nolan M. Stochastic ion channel gating in dendritic neurons: morphology dependence and probabilistic synaptic activation of dendritic spikes. PLoS Comput Biol. 2010;6:e1000886. [PMC free article] [PubMed]
3. Flight M. Synaptic transmission: On the probability of release. Nat Rev Neurosci. 2010;9:736–737.
4. Azouz R, Gray CM. Cellular mechanisms contributing to response variability of cortical neurons in vivo. J Neurosci. 1999;19:2209–2223. [PubMed]
5. GerstnerW, KistlerWM . Cambridge: Cambridge University Press; 2002. Spiking Neuron Models.
6. Brascamp JW, van Ee R, Noest AJ, Jacobs RHAH, van den Berg AV. The time course of binocular rivalry reveals a fundamental role of noise. J Vis. 2006;6:1244–1256. [PubMed]
7. Fiser J, Chiu C, Weliky M. Small modulation of ongoing cortical dynamics by sensory input during natural vision. Nature. 2004;431:573–583. [PubMed]
8. Ringach DL. Spontaneous and driven cortical activity: implications for computation. Curr Opin Neurobiol. 2009;19:1–6. [PMC free article] [PubMed]
9. Geman S, Geman D. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell. 1984;6:721–741. [PubMed]
10. Rao RPN, Olshausen BA, Lewicki MS. MIT Press; 2002. Probabilistic Models of the Brain.
11. Doya K, Ishii S, Pouget A, Rao RPN. MIT-Press; 2007. Bayesian Brain: Probabilistic Approaches to Neural Coding.
12. Körding KP, Wolpert DM. Bayesian integration in sensorimotor learning. Nature. 2004;427:244–247. [PubMed]
13. Kersten D, Mamassian P, Yuille A. Object perception as Bayesian inference. Annu Rev Psychol. 2004;55:271–304. [PubMed]
14. Gopnik A, Tenenbaum JB. Bayesian special section: Introduction; Bayesian networks, Bayesian learning and cognitive development. Dev Sci. 2007;10:281–287. [PubMed]
15. Lee TS, Mumford D. Hierarchical Bayesian inference in the visual cortex. J Opt Soc Am A. 2003;20:1434–1448. [PubMed]
16. Hoyer P, Hyvärinen A. Proceedings of the 16th Conference on Advances in Neural Information Processing Systems; December 2002. Vancouver, Canada: 2003. Interpreting neural response variability as Monte Carlo sampling of the posterior. NIPS 2002.
17. Sundareswara R, Schrater PR. Perceptual multistability predicted by search model for bayesian decisions. J Vis. 2008;8:1–19. [PubMed]
18. Gershman SJ, Vul E, Tenenbaum J. Proceedings of the 22nd Conference on Advances in Neural Information Processing Systems; December 2008. Vancouver, Canada: 2009. Perceptual multistability as Markov chain Monte Carlo inference. NIPS 2008.
19. Griffiths TL, Kemp C, Tenenbaum JB. Bayesian models of cognition. In: Sun R, editor. Handbook of Computational Cognitive Modeling. Cambridge University Press; 2008. pp. 59–100.
20. Yang T, Shadlen MN. Probabilistic reasoning by neurons. Nature. 2007;447:1075–1080. [PubMed]
21. Gold JI, Shadlen MN. The neural basis of decision making. Annu Rev Neurosci. 2007;30:535–574. [PubMed]
22. Sadaghiani S, Hesselmann G, Friston KJ, Kleinschmidt A. The relation of ongoing brain activity, evoked neural responses, and cognition. Front Syst Neurosci 4: Artikel. 2010;20 [PMC free article] [PubMed]
23. Pearl J. Morgan Kaufmann; 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference.
24. Friston KJ, Daunizeau J, Kilner J, Kiebel SJ. Action and behavior: a free-energy formulation. Biol Cybern. 2010;102:227–260. [PubMed]
25. Toussaint M. Probabilistic inference as a model of planned behavior. Künstliche Intelligenz. 2009;3:23–29.
26. Toussaint M, Goerick C. A Bayesian view on motor control and planning. In: Sigaud O, Peters J, editors. From motor to interaction learning in robots. Studies in Computational Intelligence. Springer; 2010. pp. 227–252.
27. Tenenbaum JB, Griffiths TL, Kemp C. Theory-based bayesian models of inductive learning and reasoning. Trends Cogn Sci. 2006;10:309–318. [PubMed]
28. Griffiths TL, Tenenbaum JB. Optimal predictions in everyday cognition. Psychol Sci. 2006;17:767–773. [PubMed]
29. Oaksford M, Chater N. Oxford University Press; 2007. Bayesian Rationality: The Probabilistic Approach to Human Reasoning. [PubMed]
30. Rao RPN. Neural models of Bayesian belief propagation. In: Doya K, Ishii S, Pouget A, Rao RPN, editors. Bayesian Brain. Cambridge, MA: MIT-Press; 2007. pp. 239–267.
31. Steimer A, Maass W, Douglas R. Belief-propagation in networks of spiking neurons. Neural Comput. 2009;21:2502–2523. [PubMed]
32. Deneve S. Bayesian spiking neurons I: Inference. Neural Comput. 2008;20:91–117. [PubMed]
33. Litvak S, Ullman S. Cortical circuitry implementing graphical models. Neural Comput. 2009;21:1–47. [PubMed]
34. Sahani M, Dayan P. Doubly distributional population codes: Simultaneous representation of uncertainty and multiplicity. Neural Comput. 2003;15:2255–2279. [PubMed]
35. Fiser J, Berkes P, Orbn G, Lengyel M. Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn Sci. 2010;14:119–130. [PMC free article] [PubMed]
36. Ackley DH, Hinton GE, Sejnowski TJ. A learning algorithm for boltzmann machines. Cogn Sci. 1985;9:147–169.
37. Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. Neural Comput. 2006;18:1527–1554. [PubMed]
38. Andrieu C, Freitas ND, Doucet A, Jordan MI. An introduction to MCMC for machine learning. Mach Learn. 2003;50:5–43.
39. Grimmett GR, Stirzaker DR. Oxford University Press, 3rd edition; 2001. Probability and Random Processes.
40. Nessler B, Pfeiffer M, MaassW . Proceedings of the 21th Conference on Advances in Neural Information Processing Systems; December 2008. Vancouver, Canada: 2009. Hebbian learning of Bayes optimal decisions. NIPS 2008.
41. Hinton GE. Learning to represent visual input. Philos Trans R Soc Lond B Biol Sci. 2010;365:177–184. [PMC free article] [PubMed]
42. Pillow JW, Shlens J, Paninski L, Sher A, Litke AM, et al. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature. 2008;454:995–999. [PMC free article] [PubMed]
43. Gardiner C. 3rd ed. Springer; 2004. Handbook of Stochastic Methods.
44. Leopold DA, Wilke M, Maier A, Logothetis NK. Stable perception of visually ambiguous patterns. Nat Neurosci. 2002;5:605–609. [PubMed]
45. Blake R, Logothetis NK. Visual competition. Nat Rev Neurosci. 2002;3:13–21. [PubMed]
46. Alais D, Blake R. MIT Press; 2005. Binocular Rivalry.
47. Bartels A, Logothetis NK. Binocular rivalry: a time dependence of eye and stimulus contributions. J Vis. 2010;10:3. [PubMed]
48. Binzegger T, Douglas RJ, Martin KA. A quantitative map of the circuit of cat primary visual cortex. J Neurosci. 2004;24:8441–8453. [PubMed]
49. Binzegger T, Douglas R, Martin K. Topology and dynamics of the canonical circuit of cat V1. Neural Netw. 2009;22:1071–1078. [PubMed]
50. Churchland MM, Yu BM, Cunningham JP, Sugrue LP, Cohen MR, et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat Neurosci. 2010;13:369–378. [PMC free article] [PubMed]
51. Berkes P, Orbán G, Lengyel M, Fiser J. Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science. 2011;331:83. [PMC free article] [PubMed]
52. Bishop CM. New York: Springer; 2006. Pattern Recognition and Machine Learning.
53. Koller D, Friedman N. MIT Press; 2009. Probabilistic Graphical Models: Principles and Techniques.
54. Hopfield JJ, Tank DW. “Neural” computation of decisions in optimization problems. Biol Cybern. 1985;52:141–152. [PubMed]
55. Vul E, Pashler H. Measuring the crowd within: Probabilistic representations within individuals. Psychol Sci. 2008;19:645–647. [PubMed]
56. Denison S, Bonawitz E, Gopnik A, Griffiths T. Proceedings of the 32nd Annual Conference of the Cognitive Science Society; 29 July - 1 August 2009. Amsterdam, Netherlands: 2009. Preschoolers sample from probability distributions. CogSci 2009.
57. Hinton G, Brown A. Proceedings of the 13th Conference on Advances in Neural Information Processing Systems; December 1999. Vancouver, Canada: NIPS 1999; 2000. Spiking Boltzmann machines.
58. Zemel R, Huys QJM, Natarajan R, Dayan P. Proceedings of the 17th Conference on Advances in Neural Information Processing Systems; December 2004. Vancouver, Canada: NIPS 2004; 2005. Probabilistic computation in spiking populations.
59. Kenet T, Bibitchkov D, Tsodyks M, Grinvald A, Arieli A. Spontaneously emerging cortical representations of visual attributes. Nature. 2003;425:954–956. [PubMed]
60. Fox MD, Raichle ME. Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nat Rev Neurosci. 2007;8:700–711. [PubMed]
61. Antic SD, ZhouWL, Moore AR, Short SM, Ikonomu KD. The decade of the dendritic NMDA spike. J Neurosci Res. 2010;88:2991–3001. [PubMed]
62. Merolla P, Arthur J, Shi BE, Boahen K. Expandable networks for neuromorphic chips. IEEE Trans Circuits Syst I Regul Pap. 2007;54:301–311.
63. Bruederle D, Bill J, Kaplan B, Kremkow J, Meier K, et al. Proceedings of the IEEE International Symposium on Circuits and Systems; 30May - 2 June 2010. Paris, France: ISCAS 2010; 2010. Live demonstration: Simulatorlike exploration of cortical network architectures with a mixed-signal VLSI system.
64. Python 2011. The python language reference. Available: http://docs.python.org/reference/.
65. Hinton GE. Training products of experts by minimizing contrastive divergence. Neural Comput. 2002;14:1771–1800. [PubMed]
66. Pospischil M, Piwkowska Z, Bal T, Destexhe A. Characterizing neuronal activity by describing the membrane potential as a stochastic process. J Physiol Paris. 2009;103:98–106. [PubMed]
67. Shinomoto S, Kim H, Shimokawa T, Matsuno N, Funahashi S, et al. Relating neuronal firing patterns to functional differentiation of cerebral cortex. PLoS Comput Biol. 2009;5:e1000433. [PMC free article] [PubMed]
68. Softky W, Koch C. The highly irregular firing of cortical cells is inconsistent with temporal integration of random epsps. J Neurosci. 1993;13:334–350. [PubMed]

Articles from PLoS Computational Biology are provided here courtesy of Public Library of Science
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...