{"title": "Analog Computation at a Critical Point: A Novel Function for Neuronal Oscillations?", "book": "Advances in Neural Information Processing Systems", "page_first": 137, "page_last": 144, "abstract": null, "full_text": "Analog Computation at a Critical Point: A Novel \n\nFunction for Neuronal Oscillations? \n\nLeonid Kruglyak and Willianl Bialek \n\nDepart.ment of Physics \n\nUniversity of California at Berkeley \n\nBerkeley, California 94720 \n\nand NEC Research Institute\u00b7 \n\n4 Independence vVay \n\nPrinceton, New Jersey 08540 \n\nAbstract \n\n\\Ve show that a simple spin system bia.sed at its critical point can en(cid:173)\ncode spatial characteristics of external signals, sHch as the dimensions of \n\"objects\" in the visual field. in the temporal correlation functions of indi(cid:173)\nvidual spins. Qualit.ative arguments suggest that regularly firing neurons \nshould be described by a planar spin of unit lengt.h. and such XY models \nexhibit critical dynamics over a broad range of parameters. \\Ve show how \nto extract these spins from spike trains and then mea'3ure t.he interaction \nHamilt.onian using simulations of small dusters of cells. Static correla(cid:173)\ntions among spike trains obtained from simulations of large arrays of cells \nare in agreement with the predictions from these Hamiltonians, and dy(cid:173)\nnamic correlat.ions display the predicted encoding of spatial information. \n\\Ve suggest that this novel representation of object dinwnsions in temporal \ncorrelations may be relevant t.o recent experiment.s on oscillatory neural \nfiring in the visual cortex. \n\n1 \n\nINTRODUCTION \n\nPhysical systems at a critical point exhibit long-range correlations even though \nthe interactions among the constituent partides are of short range . Through the \nfluct.uation-dissipation theorem this implies that the dynamics at one point in the \n\n\u00b7Current address. \n\n137 \n\n\f138 \n\nKruglyak and Bialek \n\nsystem are sensitive t.o external pert.urbat.ions which are applied very far away. If \nwe build a.ll analog computer poised precisely at such a critical point it should be \npossible to evaluate highly non-local funct.ionals of the input signals using a locally \ninterconnected architecture. Such a scheme would be very useful for visual compu(cid:173)\ntations, especially those which require comparisons of widely separated regions of \nthe image. From a biological point of view long-range correlat.ions at a critical point \nmight provide a robust scenario for \"responses from beyond the classical receptive \nfield\" [1]. \n\nIn this paper we present. an explicit model for analog computation at a critical \npoint and show that this model has a remarkable consequence: Because of dynamic \nscaling, spatial properties of input. signals are mapped into temporal correlat.ions \nof the local dynamics. One can, for example, measure t.he size and t.opology of \n\"object.s\" in a scene llsing only the temporal correlations in t.he output of a single \ncomputational unit (neuron) locat.ed within the object. We then show that our \nabst.ract model can be realized in networks of semi-realistic spiking neurons. The \nkey to this construction is that. neurons biased in a regime of regular or oscillatory \nfiring can be mapped to XY or planar spins [2,3]' and two-dimensional arrays of \nthese spins exhibit a broad range of parameters in which the system is generically \nat a critical point. Non-oscillatory neurons cannot, in general, be forced to operate \nat a critical point. without delicate fine tuning of the dynamics, fine tuning which \nis implausible both for biology and for man-made analog circuits. We suggest t.hat \nthese arguments may be relevant to the recent observations of oscillatory firing in \nthe visual cortex [4,5,6]. \n\n2 A STATISTICAL MECHANICS MODEL \n\n\\Ve consider a simple two-dimensional array of spins whose stat.es are defined by unit \ntwo-vect.ors Sn. These spins interact. with their neighbors so that the total energy of \nthe syst.em is H = -J L Sn ,Sm, with the sum restricted to nearest neighbor pairs. \nThis is the XY model, which is interesting in part because it possesses not a critical \npoint but rather a critical line [7] . At a given temperature, for all J > J c one finds \nthat correlations among spins decay algebraically, (Sn ,Sm) ex l/lrn - rm 117 , so that \nthere is no characteristic scale or correlation length; more precisely the correlation \nlength is infinite. In contrast, for J < J c we have (Sn,Sm) ex exp[-Irn -\nrml/{], \nwhich defines a finite correlation length {. \n\nIn the algebraic phase the dynamics of t.he spins on long length scales are rigorously \ndescribed by the spin wave approximation, in which one assumes that fluctuations \nin the angle between neighboring spins are small. In this regime it. makes sense to \nuse a continuum approximation rather than a lattice, and the energy of the system \nbecomes H = J J ({l ,z'lv 4>(x)l2, where \u00a2(x) is the orientation of the spin at position \nx. The dynamics of the syst.em are determined by the Langevin equation \n\niJ\u00a2(x,t) \n\not \n\n') \n\n= J'V-4>(x, t) + 1J(x, t), \n\nwhere I] is a Gaussian t.hermal noise source with \n\n(1J(x, i)-I](x', t')} = 2k B TcS(x - x')cS(t - I'). \n\n(1) \n\n(2) \n\n\fAnalog Computation at a Critical Point \n\n139 \n\n\\Ve can then show that the time correlation function of the spin at a single sit,(> x \nis given by \n\n(S(x. t)\u00b7S(x. 0)) = exp [-2kn TJ ~~' J (J22~k)'? 1.)- e;~:t4l. \n\n7r - w- + -' \n\n'l.ir \n\n(:3 ) \n\nIn fact Eq. 3 is valid only for an infinite array of spins. Imagine that external signals \nto this array of spins can \"activate\" and \"deactivate\" t.he spins so that one must \nreally solve Eq. 1 on finite rpgions or clusters of active spins. Then we can writ.e \nthe analog of Eq. :3 as \n\n(S(x, t)\u00b7S(x. 0)) = exp - - L-1~'71(x)I--(1- e-\n\n(4) \n\n[ knT ~ , \n\nJ \n\n7l \n\n? 1 \nAll \n\nJ>.. I I 1 \n\nn t) \u2022 \n\nwhere 1/'71 and All are the eigenfunctions and associated eigenvalues of (- v<?) on \nthe region of active spins. The key point here is that the spin auto-correlation \nfunction in time determines the spectrum of the Laplacian on the region of activity. \nBut from the classic work of I\\:ac [8] we know that this spectrum gives a great. \ndeal of information about the size and shape of the active region - we can in \ngeneral determine the area, the length of the perimeter, and the t.opology (number \nof holes) from the set of eigenvalues {An}. and this is t.rue regardless of the absolut.e \ndimensions of the region. Thus by operating at a critical point we can achieve \na scale-independent encoding of object dimension and topology in the temporal \ncorrelations of a locally connected system. \n\n3 MAPPING REAL NEURONS ONTO THE \n\nSTATISTICAL MODEL \n\nAll current models of neuralnet\",.'orks are based on the hope that most microscopic \n(\"biological\") details are unimportant for the macroscopic, collective computational \nbehavior of the system as a whole. Here we provide a rigorous connection between a \nmore realistic neural model and a simplified model with spin variables and pffective \ninteractions . essentially the XY model discussed above. A more det.ailed account is \ngiven in [2,3]. \n\n\\Ve use the Fitzhugh-Nagumo (FN) model [9.10] to describe the electrical dynamics \nof an individual neuron. This model demonstrates a threshold for firing action \npotentials. a refractory Jwriod, ano single-shot as well as repetitive firing -\nin \nshort , all tlw qualit.ative properties of neural firing. It is also known to provide a \nreasonable quant.it.at.ive df'scription of sewral cell types. To be realistic it is essential \nto add a noise current bln(t) which we take to be Gaussian, spectrally white. and \nindependent. in each cell n . \n\n\"Ve connect each neuron to its neighbors in regular one- and two-dimensional arrays. \nMore general local connections are easily added and do not significantly change t.he \nresults presented helow. We model a synapse between two neurons by exponenti(cid:173)\nating the volta.ge from one anel injecting it as current into the other. Our choice is \nmotivated by the fact that the number of t.ransmitter vesicles released at a synapse \nis exponential in the presynaptic voltage [11]; other synapt.ic transfer character(cid:173)\nistics. including sma.ll delays. give results qualitatively similar t.o those described \n\n\f140 \n\nKruglyak and Bialek \n\nhere. The resulting equations of motion are \n\n(l/Td [10 + 6In (t) -\n\n~~l(\\'~ -1) - lVn + L Jnm eXP{\\/m(t)/\\;o}] , \n\n111 \n\n(5) \n\nwhere Vn is t.he transmembrane voltage in cell 11\" 10 is the DC bias current, and t.he \nH'n are auxiliary variables; Vo sets the scale of voltage sensitivity in the synapse. \nVoltages and currents are dimensionless, and t.he parameters of the syst.em are \nexpressed in terms of the time constants TI and T2 and a dimensionless rat.io Q. \nFrom t.he voltage traces we extract the spike arrival times in the nth neuron, {til. \nWit.h the appropriate choice of parameters the FN model can be made to fire \nregularly-t.he interspike intervals are tightly clustered around a mean value. The \npower spectrum of t.he spike train s(t) Li b(t - ti) has well resolved peaks at \u00b1wo, \n\u00b12wo, .... \\Ve then low-pass filter s(1) to keep only the \u00b1wo peaks, obtaining a \nphase-modulated cosine, \n\n[Fs](t) ~ 1.410 cos[wot + \u00a2(t)], \n\n(6) \nwhere [Fs](t) denot.es the filtered spike train. By looking at [Fs](t) and its time \nderivative, we can extract the phase \u00a2(t) which describes the oscillat.ion that un(cid:173)\nderlies regular firing. Since the orientation of a planar spin is also described by a \nsingle phase variable, we can reduce the spike train to a time-dependent planar spin \nS(t). \\Ve now want to see how these spins interact when we connect two cells via \nsynapses. \n\nWe characterize the two-neuron interaction by accumulating a histogram of the \nphase differences between two connected neurons. This probability distribution \ndefines an effective Hamiltonian, P( \u00a2l, \u00a22) ex: exp[-H( \u00a2I - \u00a22)). \\Vith excitatory \nsynapses (J > 0) the interaction is ferromagnetic, as expected (sf'e Fig. 1). The \nHamiltonian takes other interesting forms for inhibitory, delayed, and nonreciprocal \nsynapses. By simulating small clusters of cells we find that interactions other than \nnearest neighbor are negligible. This leads us to predict that the entire network is \ndesc.ribed by the effective Hamiltonian H = Lij Hij(\u00a2i - \u00a2j), where Hij(\u00a2i - <Pj) \nis the effective Hamiltonian measured for the pair of connected cells i, j. \n\nOne crucial consequence of Eq. G is that correlations of the filtered spike trains \nare exactly proportional to the spin-spin correlations which are natura.l objects in \nstatistical mechanics. Specifically, if we have two cells 11, and m, \n\n( 7) \n\nThis relation shows us how the statistical description of the net.work can be tested \nin experiments which monitor actual neural spike trains. \n\n4 DOES THE MAPPING WORK? \n\n\\Vhen planar spins are connected in a one-dimensional chain with nearest-neighbor \ninteractions, correlations bet.ween spins drop off expollentially with distance. To test. \n\n\fAnalog Computation at a Critical Point \n\n141 \n\nthis prediction we have run simulations on chains of 32 Fitzhugh-Nagumo neurons \nconnected to their nearest neighbors. Correlations computeJ directly from the \nfiltered spike trains as indicated above indeed decay exponentially. as seen in the \ninsert to Fig. 1. Fig. 1 shows that the predictions for the correlation length from \nthe simple model are in excellent agreement with the correlation lengths observed \nin the simulations of spiking neurons; there are no free parameters. \n\n;~----l \n: .. \\ \n\n40 \n\n< r . ... \n~ 30 \no \n\u00a3 \n~ \n~ z.o \n~ WI \n.0 o \n\n1.0 \n\nr~ /1 \n~r \n\n\\! \nd ig \nI \n\\! \nd \nt ~_----.....3L \n\n0 \n\n... \n\nI \n\n: \n\n00 ~\n\n __ -L __ ~\n\n __ L -____ -L __ ~\n\n __ ~\n\n ____ ~\n\n0.0 \n\nr.o \n\n2.0 \n\n3.0 \n\n4.0 \n\n6.0 \n\nFigure 1: Correlation length obtained from fits to the simulation dat.a vs. correlation \nlength predicted from t.he Hamiltonians. Inset., upper left.: Correlation function vs. \ndistance from simulat.ions, with exponential fit. Inset., lower right.: Corresponding \nHamiltonian as a function of phase difference. \n\nIn t.he t.wo-dimensional case we connect each neuron to its four nearest neighbors \non a square lat.tice. The corresponding spin model is essentially the XY mode. \nHence we expect. a low-temperature (high synaptic st.rengt.h) phase wit.h correla(cid:173)\ntions that decay slowly (as a small power of distance) and a high-t.emperature (low \nsynaptic strength) disordered phase with exponential decay. These predictions were \nconfirmed by large-scale simulations of two-dimensional arrays [2]. \n\n5 OBJECT DIMENSIONS FROM TEMPORAL \n\nCORRELATIONS \n\nWe believe that we have presented convincing evidence for the description of regu(cid:173)\nla.rly firing neurons in t.erms of XY spins, at least as regards their static or equilib(cid:173)\nrium correlations. In our theoretical discussion we showed t.hat the temporal corre(cid:173)\nlation functions of XY spins in the algebraic phase contained informat.ion about the \n\n~\n \n\f142 \n\nKruglyak and Bialek \n\nhI \n\n7.7 \n\n0.0 \n\n500.0 \n\n1000.0 \n\n1500.0 \n\nFigure 2: A uto-correlation functions for the spike trains of single cells at the center \nof square arrays of different sizes. \n\ndimensions of \"objects.\" Here we test this idea in a very simple numerical exper(cid:173)\niment. Imagine that we have an a.rray of N x N connected cells which are excited \nby incoming signals so that. they are in the oscillatory regime. Obviously we can \nmeasure t.he size of this \"object\" by looking at the entire network, but. our theo(cid:173)\nretical results suggest that. one can sense these dimensions (N) using the temporal \ncorrelations in just. one cell, most simply the cell in the center of t.he array. \n\nIn Fig. 2 we show the auto-correlation functions for the spike trains of the center \nIt is clear that changing the dimensions \ncell in arrays of different dimensions. \nof the array of active cells has profound effect.s on these spa.t.iaJly local temporal \ncorrelations. Because of the fact that the model is on a critical line these correlat.ions \ncont.inue to change as the dimensions of the array increase, rather than saturating \nafter some finite correlation length is reached. Qualitatively similar results are \nexpected throughout the algebraic phase of the associated spin model. \n\nRecently it has been shown that when cells in t.he cat visual cortex are excit.ed by \nappropriat.e st.imuli t.hey enter a regime of regular firing. These firing st.atist.ics are \nsomewha.t. more complex t.han simulated here because there are a variable number \nof spikes per cycle, but we have reproduced all of our major results in models which \ncapture t.his feature of the real dat.a. We have seen that networks of regularly \nfiring cells are capable of qualitatively different types of computation because these \nnetworks can be placed at a critical point without fine tuning of paramet.ers. Most \ndramatically dynamic scaling allows us to trade spatial and t.emporal features and \nthereby encode object dimension in temporal correlations of single cells, as in Fig. \n2. To see if such novel computations are indeed mediated by cortical oscillations \n\n\fAnalog Computation at a Critical Point \n\n143 \n\nwe suggest. the direct analog of our numerical experiment, in which the correlation \nfunctions of single cells would be monitored in response to stmct.ured stimuli (e.g., \ntextures) wit.h different total spatial extent in the t.wo dimensions of the visual \nfield . 'rVe predict that these correlation functions will show a clear dependence on \nthe area of t.he visual field being excited, with some sensitivity to the shape and \ntopology as well. Most importantly this dependence on \"object\" dimension will \nextend to very large objects because the network is at a critical point. In this sense \nthe temporal correlations of single cells will encode any object dimension, rather \nthan being detectors for objects of some critical size. \n\nAcknowledgeInents \n\n\"'vVe thank O. Alvarez, D. Arovas, A. B. Bonds, K. Brueckner. M. Crair , E. \nKnobloch. H. Lecar, and D. Rohksar for helpful discussions. \n''''ork at Berkeley \nwas supported in part by the National Science Foundation through a Presidential \nYoung Investigator Award (to W.B.), supplement.ed by funds from Cray Research, \nSun Microsystems, and the NEC Research Institute , by the Fannie and John Hertz \nFoundation through a Graduate Fellowship (to L.K.), and by the USPHS through \na Biomedical Research Support Grant. \n\nReferences \n\n[1] J. Allman, F. Meizin, and E. McGuiness. Ann. Rev. Neurosci., 8:407, 1985. \n[2] L. Kruglyak. From biological reality to simple physical models: Networks of \noscillating neurons and the XY model. PhD thesis, University of California at \nBerkeley, Berkeley, California, 1990. \n\n, \n[3] W . Bialek . In E. Jen, editor, 1989 Lectures i1l CompleJ: Systems, SF! St'ud(cid:173)\nies in the Sciences of Complexity, volume 2, pages 513-595. Addison-Wesley, \nReading, Mass., 1990. \n\n[4] R. Eckhorn, R. Bauer, W. Jordan, IVI. Brosch, \"V. I{ruse, M. l\\.funk, and H. J. \n\nReit.boeck. Bioi. Cybern., 60:121, 1988. \n\n[5] C. M. Gray and W. Singer. Proc. Nat. Acad. Sci. USA, 86:1698, 1989. \n[6] C. M. Gray, P. Konig, A. K. Engel, and W. Singer. Nature, 338:334, 1989. \n[7] D. R. Nelson. In C . Domb and J. L. Lebowitz , editors, Phase Transitions and \n\nCritica.l Phenomena, volume 7, chapter 1. Academic Press , London, 1983. \n\n[8] M. Kac. Th.e American Mathematical Monthly, 73:1-23, 1966. \n[9] Richard Fitzhugh. Biophysical Journal, 1:445-466, 1961. \n[lOJ J. S. Nagumo, S. Arimoto, and S. Yoshizawa. Proc. !. R. E., 50:2061, 1962. \n[11] D. J. Aidley. The Physiology of Excitable Cells. Cambridge Universit.y Press, \n\nCambridge, 1971. \n\n\f\fPart IV \n\nTemporal Reasoning \n\n\f\f", "award": [], "sourceid": 421, "authors": [{"given_name": "Leonid", "family_name": "Kruglyak", "institution": null}, {"given_name": "William", "family_name": "Bialek", "institution": null}]}