{"title": "Synchronization, oscillations, and 1/f noise in networks of spiking neurons", "book": "Advances in Neural Information Processing Systems", "page_first": 629, "page_last": 636, "abstract": "", "full_text": "Synchronization, oscillations, and 1/ f \nnoise in networks of spiking neurons \n\nMartin Stemmler, Marius Usher, and Christof Koch \n\nPasadena, CA 91125 \n\nComputation and Neural Systems, 139-74 \n\nCalifornia Institute of Technology \n\nZeev Olami \n\nDept. of Chemical Physics \n\nWeizmann Institute of Science \n\nRehovot 76100, Israel \n\nAbstract \n\nWe investigate a model for neural activity that generates long range \ntemporal correlations, 1/ f noise, and oscillations in global activity. \nThe model consists of a two-dimensional sheet of leaky integrate(cid:173)\nand-fire neurons with feedback connectivity consisting of local ex(cid:173)\ncitation and surround inhibition. Each neuron is independently \ndriven by homogeneous external noise. Spontaneous symmetry \nbreaking occurs, resulting in the formation of \"hotspots\" of activ(cid:173)\nity in the network. These localized patterns of excitation appear \nas clusters that coalesce, disintegrate, or fluctuate in size while si(cid:173)\nmultaneously moving in a random walk constrained by the interac(cid:173)\ntion with other clusters. The emergent cross-correlation functions \nhave a dual structure, with a sharp peak around zero on top of \na much broader hill . The power spectrum associated with single \nunits shows a 1/ f decay for small frequencies and is flat at higher \nfrequencies, while the power spectrum of the spiking activity aver(cid:173)\naged over many cells-equivalent to the local field potential-shows \nno 1/ f decay but a prominent peak around 40 Hz. \n\n629 \n\n\f630 \n\nStemmler, Usher, Koch, and Olami \n\n1 The model \n\nThe model consists of a 100-by-l00 lattice of integrate-and-fire units with cyclic \nlattice boundary conditions. Each unit represents the nerve cell membrane as a \nsimple RC circuit (r = 20 msec) with the addition of a reset mechanism; the \nrefractory period TreJ is equal to one iteration step (1 msec). \n\nUnits are connected to each other within the layer by local excitatory and inhibitory \nconnections in a center-surround pattern. Each unit is excitatorily connected to \nN = 50 units chosen from a Gaussian probability distribution of u = 2.5 (in terms \nof the lattice constant), centered at the unit's position N inhibitory connections \nper unit are chosen from a uniform probability distribution on a ring eight to nine \nlattice constants away. \n\nOnce a unit reaches the threshold voltage, it emits a pulse that is transmitted in \none iteration (1 msec) to connected neighboring units, and the potential is reset by \nsubtracting the threshold from resting potential. \n\n\\Ii(t + 1) = (exp( -l/r)\\Ii(t) + h (t)) O[vth - V(t)]. \n\n(1) \nIi is the input current, which is the sum of lateral currents from presynaptic units \nand external current. The lateral current leads to an increase (decrease) in the \nmembrane potential of excitatory (inhibitorily ) connected cells. The weight of \nthe excitation and inhibition, in units of voltage threshold, is ~ and J3 ~. The \nvalues a = 1.275 and J3 = 0.67 were used for simulations. The external input is \nmodeled independently for each cell as a Poisson process of excitatory pulses of \nmagnitude 1/ N, arriving at a mean rate \"ext. Such a simple cellular model mimics \nreasonably well the discharge patterns of cortical neurons [Bernander et al., 1994, \nSoftky and Koch, 1993]. \n\n2 Dynamics and Pattern Formation \n\nIn the mean-field approximation, the firing rate of an integrate-and-fire unit is a \nfunction of the input current [Amit and Tsodyks, 1991] given by \n\nf(I) = (TreJ - r In[l - 1/(1 r)])-l, \n\n(2) \n\n(3) \n\nwhere Tref is the refractory period and r the membrane time constant. \n\nIn this approximation, the dynamics associated with eq. 1 simplify to \n\n~i = -Ii + L j Wijf(Ij) + It xt , \n\nwhere Wij represents the connection strength matrix from unit j to unit i. \nHomogeneous firing activity throughout the network will result as long as the con(cid:173)\nnectivity pattern satisfies W(k)-l < 0 for all k, where W(k) is the Fourier transform \nof Wij . As one increases the total strength of lateral connectivity, clusters of high \nfiring activity develop. These clusters form a hexagonal grid across the network; for \neven stronger lateral currents, the clusters merge to form stripes. \n\nThe transition from a homogeneous state to hexagonal clusters to stripes is generic \nto many nonequilibrium systems in fluid mechanics, nonlinear optics, reaction(cid:173)\ndiffusion systems, and biology. (The classic theory for fluid mechanics was first \n\n\fSynchronization, Oscillations, and IlfNoise in Networks of Spiking Neurons \n\n631 \n\ndeveloped by [Newell and Whitehead, 1969], see [Cross and Hohenberg, 1993] for \nan extensive review. Cowan (1982) was the first to suggest applying the techniques \nof fluid mechanics to neural systems.) \n\nThe richly varied dynamics of the model, however, can not be captured by a mean(cid:173)\nfield description. Clusters in the quasi-hexagonal state coalesce, disintegrate, or \nfluctuate in size while simultaneously moving in a random walk constrained by the \ninteraction with other clusters. \n\nR~ndom Walk of Clusters \n\n16 \n\n14 \nE 12 \n... \n\" \" \nt'.: B 8 \n; \n\n10 \n\n6 \n\no~~--~~--~~--~~--~~ \n\n14 \n\n16 \n\n18 \n\no \n\n2 \n\n6 \n12 \nx (latt~ce un~t~) \n\n10 \n\n8 \n\nFigure 1: On the left, the summed firing activity for the network over 50 msec of \nsimulation is shown. Lighter shades denote higher firing rates (maximum firing rate \n120 Hz). Note the nearly hexagonal pattern of clusters or \"hotspots\" of activity. \nOn the right, we illustrate the motion of a typical cluster. Each vertex in the graph \nrepresents a tracked cluster's position averaged over 50 msec. Repulsive interactions \nwith surrounding clusters generally constrain the motion to remain within a certain \nradius. This vibratory motion of a cluster is occasionally punctuated by longer(cid:173)\nrange diffusion. \n\nStatistical fluctuations, diffusion and synchronization of clusters, and noise in the \nexternal input driving the system lead to 1/ I-noise dynamics, long-range correla(cid:173)\ntions, and oscillations in the local field potential. These issues shall be explored \nnext. \n\n3 1/ f Noise \n\nThe power spectra of spike trains from individual units are similar to those pub(cid:173)\nlished in the literature for nonbursting cells in area MT in the behaving mon(cid:173)\nkey [Bair et al., 1994]. Power spectra were generally flat for all frequencies above \n100 Hz. The effective refractory period present in an integrate-and-fire model in(cid:173)\ntroduces a dip at low frequencies (also seen in real data). Most noteworthy is the \nl/lo.s component in the power spectrum at low frequencies. Notice that in order \nto see such a decay for very low frequencies in the spectrum, single units must be \nrecorded for on the order of 10-100 sec, longer than the recording time for a typical \ntrial in neurophysiology. \n\nWe traced a cluster of neuronal activity as it diffused through the system, and \n\n\f632 \n\nStemmler. Usher. Koch. and Olami \n\n3r-----~----~------~----_r----~ \n\nSpike Tra~n Power Spectrum \n\nlSI distribution \n\n2.5 \n\n2 \n\n1.5 \n\n1 \n\n0.5 \n\n0.7 \n0.5 \n\n... \n\n0.3 \n0.2 \n0.15 \n\n0.1 \n\n20 \n\n40 \n\nHz \n\n60 \n\n80 \n\n100 \n\n30. \n\n50. \n\n70. \n\n100. \n\n150. 200. \n\nmsec \n\nFigure 2: Typical power spectrum and lSI distribution of single units over 400 sec \nof simulation. At low frequencies, the power spectrum behaves as f- O.S\u00b1O.017 up \nto a cut-off frequency of ~ 8 Hz. The lSI distribution on the right is shown on a \nlog-log scale. The lSI histogram decays as a power law pet) ex t-1.70\u00b1O.02 between \n25 and 300 msec. In contrast, a system with randomized network connections will \nhave a Poisson-distributed lSI histogram which decays exponentially. \n\nmeasured the lSI distribution at a fixed point relative to the cluster center. In the \ncluster frame of reference, activity should remain fairly constant, so we expect and \ndo find an interspike interval (lSI) distribution with a single characteristic relaxation \ntime: \n\nPr(t) = A(r)exp(-tA(r)) , \n\nwhere the firing rate A(r) is now only a function of the distance r in cluster coordi(cid:173)\nnates. Thus Pr(t) is always Poisson for fixed r. \nIf a cluster diffuses slowly compared to the mean interspike interval, a unit at a \nfixed position samples many lSI distributions of varying A(r) as the cluster moves. \nThe lSI distribution in the fixed frame reference is thus \n\npet) = j A(r)2 exp( -t A(r\u00bb)dr. \n\n(4) \n\nDepending on the functional form of A(r), pet) (the lSI distribution for a unit at \na fixed position) will decay as a power law, and not as an exponential. Empirically, \nthe distribution of firing rates as a function of r can be approximated (roughly) by \na Gaussian. A Gaussian A(r) in eq. 4 leads to pet) f'oi t- 2 for t at long times. In \nturn, a power-law (fractal) pet) generates 1/ f noise (see Table 1). \n\n4 Long-Range Cross-Correlations \n\nExcitatory cross-correlation functions for units separated by small distances consist \nof a sharp peak at zero mean time delay followed by a slower decay characterized \nby a power law with exponent -0.21 until the function reaches an asymptotic level. \nNelson et al. (1992) found this type of cross-correlation between neurons-a \"castle \non a hill\" -to be the most common form of correlation in cat visual cortex. Inhibitory \n\n\fSynchronization, Oscillations, and lifNoise in Networks of Spiking Neurons \n\n633 \n\ncross-correlations show a slight dip that is much less pronounced than the sharp \nexcitatory peak at short time-scales. \n\nCross-Correlation at d \n\n1 \n\n1000 -\n\n750 -\n\n500 -\n\n250 \n\n1000 -\n\n750 -\n\n500 \n\n250 \n\n-300 \n\n-200 \n\n-100 \n\no \n\nmsec \n\n100 \n\n200 \n\n300 \n\nCross-Correlation at d \n\n9 \n\n-300 \n\n-200 \n\n-100 \n\no \n\nmsec \n\n100 \n\n200 \n\n300 \n\nFigure 3: Cross-correlation functions between cells separated by d units of the \nlattice. Given the center-surround geometry of connections, the upper curve corre(cid:173)\nsponds to mutually excitatory coupling and the lower to mutually inhibitory cou(cid:173)\npling. Correlations decay as l/t O.21 , consistent with a power spectrum of single \nspike trains that behaves as 1/ fo .8. \n\nSince correlations decay slowly in time due to the small exponent of the power, \nlong temporal fluctuations in the firing rate result, as the 1/ f-type power spectra of \nsingle spike trains demonstrate. These fluctuations in turn lead to high variability \nin the number of events over a fixed time period. \n\nIn fact, the decay in the auto-correlation and power spectrum, as well as the rise \nin the variability in the number of events, can be related back to the slow de(cid:173)\nIf the lSI distribution decays \ncay in the interspike interval (lSI) distribution. \nas a power law pet) ,...., t- II , then the point process giving rise to it is fractal \nwith a dimension D = v-I [Mandelbrot, 1983]. Assuming that the simula(cid:173)\ntion model can be described as a fully ergodic renewal process, all these quanti(cid:173)\nties will scale together [Cox and Lewis, 1966, Teich, 1989, Lowen and Teich, 1993, \nUsher et al., 1994]: \n\n\f634 \n\nStemmler, Usher, Koch, and Olami \n\nTable 1: Scaling Relations and Empirical Results \n\nVar(N) \n\nAuto-correlation Power Spectrum \n\nlSI Distribution \n\nVar(N) \"-J Nil \n\nA(t) \"-J t ll - 2 \n\nS(I) \"-J /-11+1 \n\npet) \"\"' t- II \n\nVar(N) '\" N1.54 \n\nA(t) \"-J t- 0 .21 \n\nS(I) \"\"' /-0.81 \n\npet) \"-J c1. 7O \n\nThese relations will be only approximate if the process is nonrenewal or nonergodic, \nor if power-laws hold over a limited range. The process in the model is clearly non(cid:173)\nrenewal, since the presence of a cluster makes consecutive short interspike intervals \nfor units within that cluster more likely than in a renewal process. Hence, we expect \nsome (slight) deviations from the scaling relations outlined above. \n\n5 Cluster Oscillations and the Local Field Potential \n\nThe interplay between the recurrent excitation that leads to nucleation of clusters \nand the \"firewall\" of inhibition that restrains activity causes clusters of high activity \nto oscillate in size. Fig 4 is the power spectrum of ensemble activity over the size \nof a typical cluster. \n\nPower Spectrum of Cluster ActlVlty withln radlus d=9 \n\n25 \n\n20 \n\n15 \n\n10 \n\n10-4 \n(lJ \n:J: \n0 \nP... \n\n5 \n\n0 \n\n0 \n\n20 \n\n40 \n\n60 \nHz \n\n80 \n\n100 \n\nFigure 4: Power spectrum of the summed spiking activity over a circular area the \nsize of a single cluster (with a radius of 9 lattice constants) recorded from a fixed \npoint on the lattice for 400 seconds. Note the prominent peak centered at 43 Hz \nand the loss of the 1// component seen in the single unit power spectra (Fig. 2). \n\nThese oscillations can be understood by examining the cross-correlations between \ncells. By the Wiener-Khinchin theorem, the power spectrum of cluster activity is the \nFourier transform of the signal's auto-correlation. Since the cluster activity is the \nsum of all single-unit spiking activity within a cluster of N cells, the autocorrelation \nof the cluster spiking activity will be the sum of N auto-correlations functions of the \n\n\fSynchronization, Oscillations, and lifNoise in Networks of Spiking Neurons \n\n635 \n\nindividual cells and N x (N - 1) cross-correlation functions among individual cells \nwithin the cluster. The ensemble activity is thus dominated by cross-correlations. \n\nIn general, the excitatory \"castles\" are sharp relative to the broad dip in the cross(cid:173)\ncorrelation due to inhibition (see Fig. 3). \nIn Fourier space, these relationships \nare reversed: broader Fourier transforms of excitatory cross-correlations are paired \nwith narrower Fourier transforms of inhibitory cross-correlations. Superposition of \nsuch transforms leads to a peak in the 30-70 Hz range and cancellation of the 1/ f \ncomponent which was present the single unit power spectrum. \n\nInterestingly, the power spectra of spike trains of individual cells within the network \n(Fig. 2) show no evidence of a peak in this frequency band. Diffusion of clusters \ndisrupts any phase relationship between single unit firing and ensemble activity. \n\nThe ensemble activity corresponds to the local field potential in neurophysiological \nrecordings. While oscillations between 30 and 90 Hz have often been seen in the \nlocal field potential (or sometimes even in the EEG) measured in cortical areas in \nthe anesthetized or awake cat and monkey, these oscillations are frequently not or \nonly weakly visible in multi- or single-unit data (e.g., [Eeckman and Freeman, 1990, \nKreiter and Singer, 1992, Gray et al., 1990, Eckhorn et al., 1993]). We here offer a \ngeneral explanation for this phenomenon. \n\nAcknowledgments: We are indebted to William Softky, Wyeth Bair, Terry Se(cid:173)\njnowski, Michael Cross, John Hopfield, and Ernst Niebur, for insightful discus(cid:173)\nsions. Our research was supported by a Myron A. Bantrell Research Fellowship, \nthe Howard Hughes Medical Institute, the National Science Foundation, the Office \nof Naval Research and the Air Force Office of Scientific Research. \n\nReferences \n\n[Amit and Tsodyks, 1991] Amit, D. J. and Tsodyks, M. V. (1991). Quantitative \nstudy of attractor neural network retrieving at low rates: 1. substrate spikes, rates \nand neuronal gain. Network Com., 2(3):259-273. \n\n[Bair et al., 1994] Bair, W., Koch, C., Newsome, W., and Britten, K. (1994). Power \nspectrum analysis of MT neurons in the behaving monkey. J. Neurosci., in press. \n[Bernander et al., 1994] Bernander, 0., Koch, C., and Usher, M. (1994). The effect \nof synchronized inputs at the single neuron level. Neural Computation, in press. \n[Cowan, 1982] Cowan, J. D. (1982). Spontaneous symmetry breaking in large scale \n\nnervous activity. Int. J. Quantum Chemistry, 22:1059-1082. \n\n[Cox and Lewis, 1966] Cox, D. and Lewis, P. A. W. (1966). The Statistical Analysis \n\nof Series of Events. Chapman and Hall, London. \n\n[Cross and Hohenberg, 1993] Cross, M. C. and Hohenberg, P. C. (1993). Pattern \n\nformation outside of equilibrium. Rev. Mod. Phys., 65(3):851-1112. \n\n[Eckhorn et al., 1993] Eckhorn, R., Frien, A., Bauer, R., Woelbern, T., and Harald, \nK. (1993). High frequency (60-90 hz) oscillations in primary visual cortex of awake \nmonkey. Neuroreport, 4:243-246. \n\n\f636 \n\nStemmler, Usher, Koch, and Olami \n\n[Eeckman and Freeman, 1990] Eeckman, F . and Freeman, W. (1990). Correlations \nbetween unit firing and EEG in the rat olfactory system. Brain Res., 528(2):238-\n244. \n\n[Grayet al., 1990] Gray, C. M., Engel, A. K., Konig, P., and Singer, W. (1990) . \nStimulus dependent neuronal oscillations in cat visual cortex: receptive field \nproperties and feature dependence. Europ. J. Neurosci., 2:607-619. \n\n[Kreiter and Singer, 1992] Kreiter, A. K. and Singer, W. (1992). Oscillatory neu(cid:173)\n\nronal responses in the visual cortex of the awake macaque monkey. Europ. J. \nNeurosci., 4:369-375. \n\n[Lowen and Teich, 1993] Lowen, S. B. and Teich, M. C. (1993). Fractal renewal \n\nprocesses generate Iff noise. Phys. Rev. E, 47(2):992-1001. \n\n[Mandelbrot, 1983] Mandelbrot, B. B. (1983). The fractal geometry of nature. W. \n\nH. Freeman, New York. \n\n[Nelson et al., 1992] Nelson, J. I., Salin, P. A., Munk, M. H.-J., Arzi, M., and \nBullier, J. (1992). Spatial and temporal coherence in cortico-cortical connections: \nA cross-correlation study in areas 17 and 18 in the cat. Visual Neuroscience, \n9:21-38. \n\n[Newell and Whitehead, 1969] Newell, A. C. and Whitehead, J. A. (1969). Finite \n\nbandwidth, finite amplitude convection. J. Fluid Mech ., 38:279-303. \n\n[Softky and Koch, 1993] Softky, W. R. and Koch, C. (1993). The highly irregular \nfiring of cortical cells is inconsistent with temporal integration of random EPSPs. \nJ. Neurosci., 13(1):334-350. \n\n[Teich, 1989] Teich, M. C. (1989). Fractal character of the auditory neural spike \n\ntrain. IEEE Trans. Biomed. Eng., 36(1):150-160. \n\n[Usher et al., 1994] Usher, M., Stemmler, M., Koch, C., and Olami, Z. (1994). Net(cid:173)\n\nwork amplification of local fluctuations causes high spike rate variability, fractal \nfiring patterns, and oscillatory local field potentials. Neural Computation, in \npress. \n\n\fPART V \n\nCONTROL, \n\nNAVIGATION, AND \n\nPLANNING \n\n\f\f", "award": [], "sourceid": 736, "authors": [{"given_name": "Martin", "family_name": "Stemmler", "institution": null}, {"given_name": "Marius", "family_name": "Usher", "institution": null}, {"given_name": "Christof", "family_name": "Koch", "institution": null}, {"given_name": "Zeev", "family_name": "Olami", "institution": null}]}