{"title": "Synaptic Transmission: An Information-Theoretic Perspective", "book": "Advances in Neural Information Processing Systems", "page_first": 201, "page_last": 207, "abstract": null, "full_text": "Synaptic Transmission: An \n\nInformation-Theoretic Perspective \n\nAmit Manwani and Christof Koch \n\nComputation and Neural Systems Program \n\nCalifornia Institute of Technology \n\nPasadena, CA 91125 \n\nemail: quixote@klab.caltech.edu \n\nkoch@klab.caltech.edu \n\nAbstract \n\nHere we analyze synaptic transmission from an infonnation-theoretic \nperspective. We derive c1osed-fonn expressions for the lower-bounds on \nthe capacity of a simple model of a cortical synapse under two explicit \ncoding paradigms. Under the \"signal estimation\" paradigm, we assume \nthe signal to be encoded in the mean firing rate of a Poisson neuron. The \nperfonnance of an optimal linear estimator of the signal then provides \na lower bound on the capacity for signal estimation. Under the \"signal \ndetection\" paradigm, the presence or absence of the signal has to be de(cid:173)\ntected. Perfonnance of the optimal spike detector allows us to compute \na lower bound on the capacity for signal detection. We find that single \nsynapses (for empirically measured parameter values) transmit infonna(cid:173)\ntion poorly but significant improvement can be achieved with a small \namount of redundancy. \n\n1 Introduction \n\nTools from estimation and infonnation theory have recently been applied by researchers \n(Bialek et. ai, 1991) to quantify how well neurons transmit infonnation about their random \ninputs in their spike outputs. In these approaches, the neuron is treated like a black-box, \ncharacterized empirically by a set of input-output records. This ignores the specific nature \nof neuronal processing in tenns of its known biophysical properties. However, a systematic \nstudy of processing at various stages in a biophysically faithful model of a single neuron \nshould be able to identify the role of each stage in infonnation transfer in tenns of the pa(cid:173)\nrameters relating to the neuron's dendritic structure, its spiking mechanism, etc. Employing \nthis reductionist approach, we focus on a important component of neural processing, the \nsynapse, and analyze a simple model of a cortical synapse under two different representa(cid:173)\n\u00b7tional paradigms. Under the \"signal estimation\" paradigm, we assume that the input signal \n\n\f202 \n\nA. Manwani and C. Koch \n\nis linearly encoded in the mean firing rate of a Poisson neuron and the mean-square error \nin the reconstruction of the signal from the post-synaptic voltage quantifies system per(cid:173)\nformance. From the performance of the optimal linear estimator of the signal, a lower \nbound on the capacity for signal estimation can be computed. Under the \"signal detection\" \nparadigm, we assume that information is encoded in an all-or-none format and the error in \ndeciding whether or not a presynaptic spike occurred by observing the post-synaptic voltage \nquantifies system performance. This is similar to the conventional absentipresent(Yes-No) \ndecision paradigm used in psychophysics. Performance of the optimal spike detector in \nthis case allows us to compute a lower bound on the capacity for signal detection. \n\n0x- 0 \n\nNoR \n\nRelease \n1 \n\nNoSpI'+ 1 K(w) 12 Smm(w) where \n). is the mean firing rate. We assume that the mean (J..Lm) and variance (CT~) of m(t) are \nchosen such that the probability that >.(t) < 0 is negligible1 The vesicle release process \nis the spike train gated by the binary channel and so it is also a Poisson process with rate \n(1 - E1 )>.(t). Since v(t) = L aih(t - ti) + n(t) is a filtered P~isson process, its power \nspectral density is given by Svv (w) =1 H(w) 12 {(J..L~+CT~)(1-E1)>'+J..L~(1-E1)21 K(w) 12 \nSmm(w)} + Snn{w). The cross-spectral density is given by the expression Svm(w) = \n(1 - Et)J..LaSmm(w)H(w)K(w). This allows us to write the mean-square error as, \n\nThus, the power spectral density ofn(t) is given by Snn = >'eff(w) + Self(w). Notice \nthat if K (w) ---+ 00, E ---+ 0 i. e. perfect reconstruction takes place in the limit of high \nfiring rates. For the parameter values chosen, SefJ{w) \u00ab >'e//(w), and can be ignored. \nConsequently, signal estimation is shot noise limited and synaptic variability increases shot \nnoise by a factor N syn = (1 + eVa2 ) / (1 - E1)' For eVa = 0.6 and E1 = 0.6, N syn = 3.4, \nand for eVa = 1 and E1 = 0.6, N syn = 5. If m(t) is chosen to be white, band-limited to \nBm Hz, closed-form expressions for E and lib can be obtained. The expression for lib is \ntedious and provides little insight and so we present only the expression for E below. \n\nE(r,BT ) = CTm [1- ~-B tan (~)l \n\n2 \n\n,1 -1 BT \n1+, T \n\n+, \n\nE is a monotonic function of, (decreasing) and BT (increasing). ,can be considered as \nthe effective number of spikes available per unit signal bandwidth and BT is the ratio of \nthe signal bandwidth and the neuron bandwidth. Plots of normalized reconstruction error \nEr = E/CT~ and llb versus mean firing rate ().) for different values of signal bandwidth Bm \nare shown in Fig. 3a and Fig. 3b respectively. Observe that lib (bits/sec) is insensitive to Bm \nfor firing rates upto 200Hz because the decrease in quality of estimation (E increases with \nBm) is compensated by an increase in the number of independent samples (2Bm) available \nper second. This phenomenon is characteristic of systems operating in the low SNR regime. \nlib has the generic form, llb = B log(1 + S/(N B)), where B, S and N denote signal \nbandwidth, signal power and noise power respectively. For low SNR, I ~ B S / (N B) = \nS / N, is independent of B. So one can argue that, for our choice of parameters, a single \nsynapse is a low SNR system. The analysis generalizes very easily to the case of multiple \nsynapses where all are driven by the same signal s (t). (Manwani and Koch, in preparation). \nHowever, instead of presenting the rigorous analysis, we appeal to the intuition gained from \nthe single synapse case. Since a single synapse can be regarded as a shot noise source, \nn parallel synapses can be treated as n parallel noise sources. Let us make the plausible \n\nlWe choose pm and O'm so that X = 30'). (std of ,X) so that Prob['x(t) ~ 0] < 0.01. \n\n\fSynaptic Transmission: An Information-Theoretic Perspective \n\n205 \n\nassumption that these noises are uncorrelated. If optimal estimation is carried out separately \nfor each synapse and the estimates are combined optimally, the effective noise variance \nis given by the harmonic mean of the individual variances i.e. l/u~eff = Li l/u~i. \nHowever, if the noises are added first and optimal estimation is carried out with respect \nto the sum, the effective noise variance is given by the arithmetic mean of the individual \n:::: Li u~dn2. If we assume that all synapses are similar so that \nvariances, i.e. u~ef f \nU~i = u 2, u~ef f = u 2 In. Plots of Er and Jib for the case of 5 identical synapses are \nshown in Fig. 3c and Fig. 3d respectively. Notice that Jib increases with Bm suggesting \nthat the system is no longer in the low SNR regime. Thus, though a single synapse has very \nlow capacity, a small amount of redundancy causes a considerable increase in performance. \nThis is consistent with the fact the in the low S N R regime, J increases linearly with S N R , \nconsequently, linearly with n, the number of synapses. \na) \n\nb) \n\nx x \n\n0 \n\n0 \nx x x x \n\n0 ~ ~ + ~ .. + \n\n00 0 00 0 0 \n\n.. + + + + \n0 \n\n0 \n\n0 \n\no.a \n\nx x x \n\nx x x \n\nX X X \n\nX X X \n\nx \no \n\nB = 10Hz \nm \nBm- 25Hz \n\nBm= 50 Hz \n\n-\n\n-\n\n-\n\nBm=75HZ \nBm: 100Hz \n\n1. \n\n12 \n\nx \no \n\n-\n\n-\n\n-\n\nB = 10Hz \nm \nB=25Hz \nm \n\nBm= 50Hz \nBm=75Hz \nBm= 100 Hz \n\no.s \n\n~ e \nW 0.7 \n\"0 \nQ)m \n.~ 1U 0.8 \nE ~ o \n\n0.5 \n\nZ \n\no.a \n\n'- 0.8 g \n\nW \n\"0 0.7 \nQ) \n.~ \n(Q o.s \n\nE o \n\nZ 0.5 \n\n0 .\u2022 \n\n20 \n\n40 \n\nl1li \n\nl1li \n\n100 \n\n120 \n\n140 \n\n1l1li \n\n180 \n\n200 \n\nl1li \n\n80 \n\n100 \n\n120 \n\n140 \n\n180 \n\n180 \n\n200 \n\n~ . \nx 0 \n\n.. +\"-: .... __ .. \n\n0 \n\n+ + \n\n.. \n\n-\n\n+ + + - - - - - - : (cid:173)\n\n.. . . \n\n.. + + + \n\no \n\n0 \n\no 0 \n\no 0 \n\n1. \n\n12 \n\nI \n\nQ) \n\nU IO \n(/) UiS \n.\"t::: \n.0 \n-\nQ) \n\nS \n\n~. \n.E2 \n.5 \n\nFiring Rate (Hz) \n\n20 \n\n40 \n\n80 ~ ~ ~ ~ ~ -\n\nl1li \nFiring Rate (Hz) \n\nFigure 3: Er and!,b vs. mean firing rate (X) for n = I [(a) and (b)] and n = 5 [(c) and (d)] identical \nsynapses respectively (different values of Em) for signal estimation. Parameter values are 101 = 0.6, \n100 = 0, eVa = 0.6, ts = 0.5 msec, T = I Omsec, (7n = 0.1 mY, En = 100 Hz. \n\n4 Signal Detection \n\nThe goal in signal detection is to decide which member from a finite set of signals was \ngenerated by a source, on the basis of measurements related to the output only in a statistical \nsense. Our example corresponds to its simplest case, that of binary detection. The objective \nis to derive an optimal spike detector based on the post-synaptic voltage in a given time \ninterval. The criterion of optimality is minimum probability of error (Pe ). A false alarm \n\n\f206 \n\nA. Manwani and C. Koch \n\n(FA) error occurs when a spike is falsely detected even when no presynaptic spike occurs \nand a miss error (M) occurs when a spike fails to be detected. The probabilities of the errors \nare denoted by P, and Pm respectively. Thus, Pe = (1- Po) P, +Po Pm where Po denotes \nthe a priori probability of a spike occurrence. Let X and Y be binary variables denoting \nspike occurrence and the decision respectively. Thus, X = 1 if a spike occurred else X = \nO. Similarly, Y = 1 expresses the decision that a spike occurred. The posterior likelihood \nratio is defined as \u00a3(v) = Pr(v I X = l)/Pr(v I X = 0) and the prior likelihood as \n\u00a30 = (1 - Po)/Po. The optimal spike detector employs the well-known likelihood ratio \ntest, \"If\u00a3(v) ~ \u00a30 Y=lelseY=O\". When X = 1,v(t) = ah(t)+n(t) elsev(t) = n(t). \nSince a is a random variable, \u00a3(v) = (f Pr(v I X = 1; a) P(a) da)/ Pr(v I X = 0). If \nthe noise n( t) is Gaussian and white, it can be shown that the optimal decision rule reduces \nto a matchedfilte?, i.e. if the correlation, r between v(t) and h(t) exceeds a particular \nthreshold (denoted by TJ), Y = 1 else Y = O. The overall decision system shown in \nFig. 1 can be treated as effective binary channel (Fig. 2b). The system perfonnance can \nbe quantified either by Pe or J (X; Y), the mutual infonnation between the binary random \nvariables, X and Y. Note that even when n(t) = 0 (SN R = 00), Pe =j:. 0 due to the \nunreliability of vesicular release. Let Pe* denote the probability of error when S N R = 00. \nIf EO = 0, Pe* = Po El is the minimum possible detection error. Let PJ and P~ denote FA \nand M errors when the release is ideal (El = 0, EO = 0). It can be shown that \n\nPe = Pe* + P~[Po(1- Ed -\n\n(1 - Po)EO] + PJ[(l - Po)(l - EO) - PoEl] \n\nP, = PJ ' Pm = P~ + El (1 - P~ + PI) \n\nBoth PJ and P~ depend on TJ. The optimal value ofT) is chosen such that Pe is minimized. \nIn general, PJ and P~ can not be expressed in closed-fonn and the optimal 'f} is found using \nthe graphical ROC analysis procedure. Ifwe normalize a such that /-La = 1, PJ and P~ can \nbe parametrically expressed in tenns ofa nonnalized threshold 'f}*, PJ = 0.5[1- Er f('f}*)], \nP~ = 0.5[1+ Iooo Erf(TJ* - JSNRa) P(a) da]. J(X;Y) can be computed using the \nfonnula for the mutual infonnation for a binary channel, J = 1i (Po (1 - Pm) + (1 -\nPo) P,) - Po 1i(Pm ) - (1- Po)1i(P, ) where 1i(x) = -x log2 (x) - (1- x) log2(1- x) is \nthe binary entropy function. The analysis can be generalized to the case of n syna!Jses but \nthe expressions involve n-dimensional integrals which need to be evaluated numerically. \nThe Central Limit Theorem can be used to simplify the case of very large n. Plots of \nPe and J(X; Y) versus n for different values of SNR (1,10,00) for the case of identical \nsynapses are shown in Fig. 4a and Fig. 4b respectively. Yet again, we observe the poor \nperfonnance of a single synapse and the substantial improvement due to redundancy. The \nlinear increase of J with n is similar to the result obtained for signal estimation. \n\n5 Conclusions \n\nWe find that a single synapse is rather ineffective as a communication device but with \na little redundancy neuronal communication can be made much more robust. Infact, a \nsingle synapse can be considered as a low SNR device, while 5 independent synapses \nin parallel approach a high SNR system. This is consistently echoed in the results for \nsignal estimation and signal detection. The values of infonnation rates we obtain are very \nsmall compared to numbers obtained from some peripheral sensory neurons (Rieke et. ai, \n1996). This could be due to an over-conservative choice of parameter values on our part \nor could argue for the preponderance of redundancy in neural systems. What we have \npresented above are preliminary results of work in progress and so the path ahead is much \n\n2 For deterministic a, the result is well-known, but even if a is a one-sided random variable, the \n\nmatched filter can be shown to be optimal. \n\n\fSynaptic Tranrmission: An lnformation-Theoretic Perspective \n\n207 \n\nb) \n\n-4-- SNR = In!. \n..... SNR=10 \n--SNR= 1 \n\n.\"r;:====~\"'-'-------::::::::~ \n\na) \n\n~ e w 0. \n\n0.' \n\n0 ... \n\n-4-- SNR = In!. \n..... SNR=10 \n--SNR=1 \n\ni' 0.7 \n~ 0 \u2022\u2022 \n~ ... \n:0 \n* ex: 0.4 \nc: \n.~ 0.:1 \nE \n\u00a3 .., \n\n00.2 \n\no~, --~~2~~--~3--~--~'--~~ \n\nNumber of Synapses (n) \n\n~~~--~2~----~3------~'----~ \n\nNumber of Synapses (n) \n\nPe (a) and l,b (b) vs. the number of synapses, n, (different values of SN R) for signal detection. \nSNR = Inf. corresponds to no post-synaptic voltage noise. All the synapses are assumed to be \nidentical. Parameter values are po = 0.5, 101 = 0.6, 100 = 0, eVa = 0.6, ts = 0.5 msec, T = 10 msec, \nan = 0.1 mY, Bn = 100 Hz. \n\nlonger than the distance we have covered so far. To the best of our knowledge, analysis \nof distinct individual components of a neuron from an communications standpoint has not \nbeen carried out before. \n\nAcknowledgements \n\nThis research was supported by NSF, NIMH and the Sloan Center for Theoretical Neuro(cid:173)\nscience. We thank Fabrizio Gabbiani for illuminating discussions. \n\nReferences \n\nBekkers, J.M., Richerson, G.B. and Stevens, C.F. (1990) \"Origin of variability in quantal \nsize in cultured hippocampal neurons and hippocampal slices,\" Proc. Natl. Acad. Sci. USA \n87: 5359-5362. \nBialek, W. Rieke, F. van Steveninck, R.D.R. and Warland, D. (1991) \"Reading a neural \ncode,\" Science 252: 1854-1857. \n\nCover, T.M., and Thomas, lA. (1991) Elements of Information Theory. New York: Wiley. \n\nKom, H. and Faber, D.S. (1991) \"Quantal analysis and synaptic efficacy in the CNS,\" \nTrends Neurosci. 14: 439-445. \n\nMarkram, H. and Tsodyks, T. (1996) \"Redistibution of synaptic efficacy between neocorti(cid:173)\ncal pyramidal neurons,\" Nature 382: 807-810. \n\nRieke, F. Warland, D. van Steveninck, R.D.R. and Bialek, W. (1996) Spikes: Exploring the \nNeural Code. Cambridge: MIT Press. \n\nStevens, C.F. (1994) \"What form should a cortical theory take,\" In: Large-Scale Neuronal \nTheories of the Brain, Koch, C. and Davis, J.L., eds., pp. 239-256. Cambridge: MIT Press. \n\n\f", "award": [], "sourceid": 1477, "authors": [{"given_name": "Amit", "family_name": "Manwani", "institution": null}, {"given_name": "Christof", "family_name": "Koch", "institution": null}]}