Part of Advances in Neural Information Processing Systems 10 (NIPS 1997)
Wolfgang Maass, Anthony Zador
In most neural network models, synapses are treated as static weights that change only on the slow time scales of learning. In fact, however, synapses are highly dynamic, and show use-dependent plasticity over a wide range of time scales. Moreover, synaptic transmission is an inherently stochastic process: a spike arriving at a presynaptic terminal triggers release of a vesicle of neurotransmitter from a release site with a probability that can be much less than one. Changes in release probability represent one of the main mechanisms by which synaptic efficacy is modulated in neural circuits. We propose and investigate a simple model for dynamic stochastic synapses that can easily be integrated into common models for neural computation. We show through computer simulations and rigorous theoretical analysis that this model for a dynamic stochastic synapse increases computational power in a nontrivial way. Our results may have implications for the process(cid:173) ing of time-varying signals by both biological and artificial neural networks.
A synapse 8 carries out computations on spike trains, more precisely on trains of spikes from the presynaptic neuron. Each spike from the presynaptic neuron mayor may not trigger the release of a neurotransmitter-filled vesicle at the synapse. The probability of a vesicle release ranges from about 0.01 to almost 1. Furthermore this release probability is known to be strongly "history dependent" [Dobrunz and Stevens, 1997]. A spike causes an excitatory or inhibitory potential (EPSP or IPSP, respectively) in the postsynaptic neuron only when a vesicle is released.
A spike train is represented as a sequence 1 of firing times, i.e. as increasing sequences of numbers tl < t2 < ... from R+ := {z E R: z ~ O} . For each spike train 1 the output of synapse 8 consists of the sequence 8W of those ti E 10n which vesicles are "released" by 8 , i.e. of those t, E 1 which cause an excitatory or inhibitory postsynaptic potential (EPSP or IPSP, respectively). The map 1 -+ 8(1) may be viewed as a stochastic function that is computed by synapse S. Alternatively one can characterize the output SW of a synapse 8 through its release pattern q = qlq2 ... E {R, F}· , where R stands for release and F for failure of release. For each t, E 1 one sets q, = R if ti E 8(1) , and qi = F if ti ¢ 8W .
Dynamic Stochastic Synapses as Computational Units
195