{"title": "A neural network implementing optimal state estimation based on dynamic spike train decoding", "book": "Advances in Neural Information Processing Systems", "page_first": 145, "page_last": 152, "abstract": null, "full_text": "A neural network implementing optimal state\n\nestimation based on dynamic spike train decoding\n\nOmer Bobrowski1, Ron Meir1, Shy Shoham2 and Yonina C. Eldar1\nDepartment of Electrical Engineering1 and Biomedical Engineering2\n\nTechnion, Haifa 32000, Israel\n\n{bober@tx},{rmeir@ee},{sshoham@bm},{yonina@ee}.technion.ac.il\n\nAbstract\n\nIt is becoming increasingly evident that organisms acting in uncertain dynamical\nenvironments often employ exact or approximate Bayesian statistical calculations\nin order to continuously estimate the environmental state, integrate information\nfrom multiple sensory modalities, form predictions and choose actions. What is\nless clear is how these putative computations are implemented by cortical neural\nnetworks. An additional level of complexity is introduced because these networks\nobserve the world through spike trains received from primary sensory afferents,\nrather than directly. A recent line of research has described mechanisms by which\nsuch computations can be implemented using a network of neurons whose activ-\nity directly represents a probability distribution across the possible \u201cworld states\u201d.\nMuch of this work, however, uses various approximations, which severely re-\nstrict the domain of applicability of these implementations. Here we make use of\nrigorous mathematical results from the theory of continuous time point process\n\ufb01ltering, and show how optimal real-time state estimation and prediction may be\nimplemented in a general setting using linear neural networks. We demonstrate\nthe applicability of the approach with several examples, and relate the required\nnetwork properties to the statistical nature of the environment, thereby quantify-\ning the compatibility of a given network with its environment.\n\n1 Introduction\n\nA key requirement of biological or arti\ufb01cial agents acting in a random dynamical environment is\nestimating the state of the environment based on noisy observations. While it is becoming clear that\norganisms employ some form of Bayesian inference, it is not yet clear how the required computa-\ntions may be implemented in networks of biological neurons. We consider the problem of a system,\nreceiving multiple state-dependent observations (possibly arising from different sensory modalities)\nin the form of spike trains, and construct a neural network which, based on these noisy observations,\nis able to optimally estimate the probability distribution of the hidden world state.\n\nThe present work continues a line of research attempting to provide a probabilistic Bayesian frame-\nwork for optimal dynamical state estimation by biological neural networks. In this framework, \ufb01rst\nformulated by Rao (e.g., [8, 9]), the time-varying probability distributions are represented in the\nneurons\u2019 activity patterns, while the network\u2019s connectivity structure and intrinsic dynamics are\nresponsible for performing the required computation. Rao\u2019s networks use linear dynamics and dis-\ncrete time to approximately compute the log-posterior distributions from noisy continuous inputs\n(rather than actual spike trains). More recently, Beck and Pouget [1] introduced networks in which\nthe neurons directly represent and compute the posterior probabilities (rather than their logarithms)\nfrom discrete-time approximate \ufb01ring rate inputs, using non-linear mechanisms such as multiplica-\ntive interactions and divisive normalization. Another relevant line of work, is that of Brown and\ncolleagues as well as others (e.g., [4, 11, 13]) where approximations of optimal dynamical estima-\n\n1\n\n\ftors from spike-train based inputs are calculated, however, without addressing the question of neural\nimplementation.\n\nOur approach is formulated within a continuous time point process framework, circumventing many\nof the dif\ufb01culties encountered in previous work based on discrete time approximations and input\nsmoothing. Moreover, using tools from the theory of continuous time point process \ufb01ltering (e.g.,\n[3]), we are able to show that a linear system suf\ufb01ces to yield the exact posterior distribution for\nthe state. The key element in the approach is switching from posterior distributions to a new set\nof functions which are simply non-normalized forms of the posterior distribution. While posterior\ndistributions generally obey non-linear differential equations, these non-normalized functions obey\na linear set of equations, known as the Zakai equations [15]. Intriguingly, these linear equations\ncontain the full information required to reconstruct the optimal posterior distribution! The linearity\nof the exact solution provides many advantages of interpretation and analysis, not least of which is\nan exact solution, which illustrates the clear distinction between observation-dependent and inde-\npendent contributions. Such a separation leads to a characterization of the system performance in\nterms of prior knowledge and real-time observations. Since the input observations appear directly\nas spike trains, no temporal information is lost. The present formulation allows us to consider in-\nputs arising from several sensory modalities, and to determine the contribution of each modality to\nthe posterior estimate, thereby extending to the temporal domain previous work on optimal multi-\nmodal integration, which was mostly restricted to the static case. Inherent differences between the\nmodalities, related to temporal delays and different shapes of tuning curves can be incorporated and\nquanti\ufb01ed within the formalism.\n\nIn a historical context we note that a mathematically rigorous approach to point process based \ufb01l-\ntering was developed during the early 1970s following the seminal work of Wonham [14] for \ufb01nite\nstate Markov processes observed in Gaussian noise, and of Kushner [7] and Zakai [15] for diffusion\nprocesses. One of the \ufb01rst papers presenting a mathematically rigorous approach to nonlinear \ufb01l-\ntering in continuous time based on point process observations was [12], where the exact nonlinear\ndifferential equations for the posterior distributions are derived. The presentation in Section 4 sum-\nmarizes the main mathematical results initiated by the latter line of research, adapted mainly from\n[3], and serves as a convenient starting point for many possible extensions.\n\n2 A neural network as an optimal \ufb01lter\n\nConsider a dynamic environment characterized at time t by a state Xt, belonging to a set of N\nstates, namely Xt \u2208 {s1, s2, . . . , sN }. We assume the state dynamics is Markovian with genera-\ntor matrix Q. The matrix Q, [Q]ij = qij, is de\ufb01ned [5] by requiring that for small values of h,\nPr[Xt+h = si|Xt = si] = 1 + qiih + o(h) and Pr[Xt+h = sj|Xt = si] = qijh + o(h) for j 6= i.\n\nThe normalization requirement is thatPj qij = 0. This matrix controls the process\u2019 in\ufb01nitesimal\n\nprogress according to \u02d9\u03c0(t) = \u03c0(t)Q, where \u03c0i(t) = Pr[Xt = si].\nThe state Xt is not directly observable, but is only sensed through a set of M random state-dependent\nobservation point processes {N (k)\nto represent the spiking\nactivity of the k-th sensory cell, and assume these processes to be doubly stochastic Poisson counting\nprocesses1 with state-dependent rates \u03bbk(Xt). These processes are assumed to be independent,\ngiven the current state Xt. The objective of state estimation (a.k.a. nonlinear \ufb01ltering) is to obtain a\ndifferential equation for the posterior probabilities\n\nk=1. We take each point process N (k)\n\nt }M\n\nt\n\npi(t)\n\n4\n\n= PrhXt = si(cid:12)(cid:12)(cid:12)\n\n(1)\n\nN (1)\n\n[0,t] i ,\n[0,t], . . . , N (M )\n= nN (1)\n\n4\n\n[0,t] = {N (k)\n\ns }0\u2264s\u2264t. In the sequel we denote Y t\n0\n\nwhere N (k)\nreader to Section 4 for precise mathematical de\ufb01nitions.\nWe interpret the rate \u03bbk as providing the tuning curve for the k-th sensory input. In other words,\nthe k-th sensory cell responds with strength \u03bbk(si) when the input state is Xt = si. The required\ndifferential equations for pi(t) are considerably simpli\ufb01ed, with no loss of information [3], by con-\nj=1 \u03c1j(t).\n\nsidering a set of non-normalized \u2018probability functions\u2019 \u03c1i(t), such that pi(t) = \u03c1i(t)/PN\n\n[0,t]o, and refer the\n\n1Implying that the rate function itself is a random process.\n\n[0,t], . . . , N (M )\n\n2\n\n\fBased on the theory presented in Section 4 we obtain\n\n\u02d9\u03c1i(t) =\n\nN\n\nXj=1\n\nQji\u03c1j(t) +\n\nM\n\nXk=1\n\n(\u03bbk(si) \u2212 1)\"Xn\n\n\u03b4(t \u2212 tk\n\nn) \u2212 1# \u03c1i(t),\n\n(2)\n\nwhere {tk\nform by de\ufb01ning\n\nn} denote the spiking times of the k-th sensory cell. This equation can be written in vector\n\n\u039bk = diag(\u03bbk(s1) \u2212 1, \u03bbk(s2) \u2212 1, . . . \u03bbk(sN ) \u2212 1)\n\n;\n\n\u039b =\n\nand \u03c1 = (\u03c11, . . . , \u03c1N ), leading to\n\n\u039bk,\n\n(3)\n\nM\n\nXk=1\n\n\u02d9\u03c1(t) = (Q \u2212 \u039b)>\n\n\u03c1(t) +\n\nM\n\nXk=1\n\n\u039bkXn\n\n\u03b4(t \u2212 tk\n\nn)\u03c1(t).\n\n(4)\n\nEquations (2) and (4) can be interpreted as the activity of a linear neural network, where \u03c1i(t)\nrepresents the \ufb01ring rate of neuron i at time t, and the matrix (Q \u2212 \u039b)> represents the synaptic\nweights (including self-weights); see Figure 1 for a graphical display of the network. Assuming\nthat the tuning functions \u03bbk are unimodal, decreasing in all directions from some maximal value\n(e.g., Gaussian or truncated cosine functions), we observe from (2) that the impact of an input spike\nat time t is strongest on cell i for which \u03bbk(si) is maximal, and decreases signi\ufb01cantly for cells\nj for which sj is \u2018far\u2019 from si. This effect can be modelled using excitatory/inhibitory connec-\ntions, where neurons representing similar states excite each other, while neurons corresponding to\nvery different states inhibit each other (e.g., [2]). This issue will be elaborated on in future work.\n\nSeveral observations are in place regarding (4).\n(i) The so-\nlution of (4) provides the optimal posterior state estimator\ngiven the spike train observations, i.e., no approximation is in-\nvolved. (ii) The equations are linear even though the equations\nobeyed by the posterior probabilities pi(t) are nonlinear. (iii)\nThe temporal evolution breaks up neatly into an observation-\nindependent term, which can be conceived of as implementing\na Bayesian dynamic prior, and an observation-dependent term,\nwhich contributes each time a spike occurs. Note that a simi-\nlar structure was observed recently in [1]. (iv) The observation\nprocess affects the posterior estimate through two terms. First,\ninput processes with strong spiking activity, affect the activity\nmore strongly. Second, the k-th input affects most strongly the\ncomponents of \u03c1(t) corresponding to states with large values\nof the tuning curve \u03bbk(si). (v) At this point we assume that the\nmatrix Q is known. In a more general setting, one can expect\nQ to be learned on a slower time scale, through interaction\nwith the environment. We leave this as a topic for future work.\nMulti-modal inputs A multi-modal scenario may be envisaged as one in which a subset of the sen-\nsory inputs arises from one modality (e.g., visual) while the remaining inputs arise from a different\nsensory modality (e.g., auditory). These modalities may differ in the shapes of their receptive \ufb01elds,\ntheir response latencies, etc. The framework developed above is suf\ufb01ciently general to deal with\nany number of modalities, but consider for simplicity just two modalities, denoted by V and A. It is\nstraightforward to extend the derivation of (4), leading to\n\nFigure 1: A graphical depiction of\nthe network implementing optimal\n\ufb01ltering of M spike train inputs.\n\n\u02d9\u03c1(t) = (Q \u2212 \u039bv \u2212 \u039ba)>\n\n\u03c1(t) +( Mv\nXk=1\n\n\u039bv\n\nkXn\n\n\u03b4(t \u2212 tv,k\n\nn ) +\n\nMa\n\nXk=1\n\n\u039ba\n\nkXn\n\n\u03b4(t \u2212 ta,k\n\nn )) \u03c1(t).\n\n(5)\n\nPrediction The framework can easily be extended to prediction, de\ufb01ned as the problem of calcu-\nlating the future posterior distribution ph\nIt is easy to show that the\n\n0 ].\ni (t) = Pr[Xt+h = si|Y t\n\n3\n\n\fnon-normalized probabilities \u03c1\n\nh(t) can be calculated using the vector differential equation\n\nM\n\nh(t) = (Q \u2212 \u02dc\u039b)>\n\u02d9\u03c1\n\nh(t) +\n\n\u03c1\n\n\u03b4(t \u2212 tk\n\nn)\u03c1\n\nh(t),\n\n(6)\n\nXk=1\n\n\u02dc\u039bkXn\n\nh(0) = ehQ>\n\n\u039bke\u2212hQ>. Interestingly, the\n\n\u03c1(0), and where \u02dc\u039bk = ehQ>\n\nwith the initial condition \u03c1\nequations obtained are identical to (4), except that the system parameters are modi\ufb01ed.\nSimpli\ufb01ed equation When the tuning curves of the sensory cells are uniformly distributed Gaus-\nsians (e.g., spatial receptive \ufb01elds), namely \u03bbk(x) = \u03bbmax exp(\u2212(x\u2212k\u2206x)2/2\u03c32), it can be shown\nk=1 \u03bbk(x) \u2248 \u03b2 for all x, im-\n\n[13] that for small enough \u2206x, and a large number of sensory cells,PM\nplying that \u039b = Pk \u039bk \u2248 (\u03b2 \u2212 M )I. Therefore the matrix \u039b has no effect on the solution of (4),\n\nexcept for an exponential attenuation that is applied to all the cells simultaneously. Therefore, in\ncases where the number of sensory cells is large, \u039b can be omitted from (4). This means that be-\ntween spike arrivals, the system behaves solely according to the a-priori knowledge about the world,\nand when a spike arrives, this information is reshaped according to the \ufb01ring cell\u2019s tuning curve.\n\nYi=1\n\n3 Theoretical Implications and Applications\n\nViewing (4) we note that between spike arrivals, the input has no effect on the system. Therefore,\nthe inter-arrival dynamics is simply \u02d9\u03c1(t) = (Q \u2212 \u039b)>\n\u03c1(t). De\ufb01ning tn as the n-th arrival time of a\nspike from any one of the sensors, the solution in the interval (tn, tn+1) is\n\nWhen a new spike arrives from the k-th sensory neuron at time tn the system is modi\ufb01ed within an\nin\ufb01nitesimal window of time as\n\n\u03c1(t) = e(t\u2212tn)(Q\u2212\u039b)>\n\n\u03c1(tn).\n\n\u03c1i(t+\n\nn ) = \u03c1i(t\u2212\n\nn ) + \u03c1i(t\u2212\n\nn )(\u03bbk(si) \u2212 1) = \u03c1i(t\u2212\n\nn )\u03bbk(si).\n\n(7)\n\nThus, at the exact time of a spike arrival from the k-th sensory cell, the vector \u03c1 is reshaped according\nto the tuning curve of the input cell that \ufb01red this spike. Assuming n spikes occurred before time t,\nwe can derive an explicit solution to (4), given by\n\nn\n\n\u03c1(t) = e(t\u2212tn)(Q\u2212\u039b)>\n\n(I + \u039bk(ti))e(ti\u2212ti\u22121)(Q\u2212\u039b)>\n\n\u03c1(0),\n\n(8)\n\nwhere k(ti) is the index of the cell that \ufb01red at ti, I is the identity matrix, and we assumed initial\nconditions \u03c1(0) at t0 = 0.\n\n3.1 Demonstrations\n\nWe demonstrate the operation of the system on several synthetic examples. First consider a small\nobject moving back and forth on a line, jumping between a set of discrete states, and being ob-\nserved by a retina with M sensory cells. Each world state si describes the particle\u2019s position,\nand each sensory cell k generates a Poisson spike train with rate \u03bbk(Xt), taken to be a Gaussian\n\u03bbmax exp (\u2212(x \u2212 xk)2/2\u03c32). Figure 2(a) displays the motion of the particle for a speci\ufb01c choice of\nmatrix Q, and 2(b) presents the spiking activity of 10 position sensitive sensory cells. Finally, Figure\n2(c) demonstrates the tracking ability of the system, where the width of the gray trace corresponds\nto the prediction con\ufb01dence. Note that the system is able to distinguish between 25 different states\nrather well with only 10 sensory cells.\nIn order to enrich the systems\u2019s estimation capabilities, we can include additional parameters within\nthe state of the world. Considering the previous example, we create a larger set of states: \u02dcsij =\n(si, dj), where dj denotes the current movement direction (in this case d1=up, d2=down). We add\na population of sensory cells that respond differently to different movement directions. This lends\nfurther robustness to the state estimation. As can be seen in Figure 2(d)-(f), when for some reason the\ninput of the sensory cells is blocked (and the sensory cells \ufb01re spontaneously) the system estimates\na movement that continues in the same direction. When the blockade is removed, the system is re-\nsynchronized with the input. It can be seen that even during periods where sensory input is absent,\nthe general trend is well predicted, even though the estimated uncertainty is increased.\n\n4\n\n\fBy expanding the state space it is also possible for the system to track multiple objects simultane-\nously. In \ufb01gure 2(g)-(i) we present tracking of two simultaneously moving objects. This is done\nsimply by creating a new state space, sij = (s1\ni denotes the state (location) of the\nk-th object.\n\nj ), where sk\n\ni , s2\n\n(a) Object trajectory\n\n(d) Object trajectory\n\n(g) Object trajectory\n\n20\n\n10\n\n]\n\nm\nc\n[\nx\n\n0\n0\n\n10\n\n5\n\n0\n0\n\n#\n\n \nl\nl\n\ne\nc\n\n25\n20\n15\n10\n5\n\n]\n\nm\nc\n[\nx\n\n1\n\n2\n\n3\n\n4\n\n5\n\nt[sec]\n\n6\n\n7\n\n8\n\n9\n\n(b) Input activity\n\n1\n\n2\n\n3\n\n4\n\n5\n\nt[sec]\n\n6\n\n7\n\n8\n\n9\n\n(c) Posterior probability evolution\n\n20\n\n10\n\n]\n\nm\nc\n[\nx\n\n0\n0\n\n#\n\n \nl\nl\n\ne\nc\n\n10\n\n5\n\n0\n0\n\n25\n20\n15\n10\n5\n\n]\n\nm\nc\n[\nx\n\n1\n\n2\n\n3\n\n4\n\n5\n\nt[sec]\n\n6\n\n7\n\n8\n\n9\n\n(e) Input activity\n\n1\n\n2\n\n3\n\n4\n\n5\n\nt[sec]\n\n6\n\n7\n\n8\n\n9\n\n]\n\nm\nc\n[\nx\n\n#\n\n \nl\nl\n\ne\nc\n\n10\n\n5\n\n0\n \n0\n\n10\n\n5\n\n0\n0\n\n \n\nObject #1\nObject #2\n\n1\n\n2\n\n3\n\n4\n\n5\n\nt[sec]\n\n6\n\n7\n\n8\n\n9\n\n(h) Input activity\n\n1\n\n2\n\n3\n\n4\n\n5\n\nt[sec]\n\n6\n\n7\n\n8\n\n9\n\n(f) Posterior probability evolution\n\n(i) Posterior probability evolution\n\n]\n\nm\nc\n[\nx\n\n10\n8\n6\n4\n2\n0 \n\n2\n\n4\n\nt[sec]\n\n6\n\n8\n\n10\n\n0 \n\n2\n\n4\n\nt[sec]\n\n6\n\n8\n\n10\n\n0 \n\n2\n\n4\n\nt[sec]\n\n6\n\n8\n\n10\n\nFigure 2: Tracking the motion of an object in 1D. (a) The object\u2019s trajectory. (b) Spiking activity\nof 10 sensory cells.\n(c) Decoded position estimation with con\ufb01dence interval. Each of the 10\nsensory cells has a Gaussian tuning curve of width \u03c3 = 2 and maximal \ufb01ring rate \u03bbmax = 25.(d)-(f)\nTracking based on position and direction information. The red bar marks the time when the input\nwas blocked, and the green bar marks the time when the blockade was removed. Here we used 10\nplace-cells and 4 direction-cells (marked in red). (g)-(i) Tracking of two objets simultaneously. The\n\nnetwork activity in (i) represents Pr(cid:2)X 1\n\n3.2 Behavior Characterization\n\nt = si \u2228 X 2\n\nt = si|Y t\n\n0(cid:3).\n\nThe solution of the \ufb01ltering equations (4) depends on two processes, namely the recurrent dynamics\ndue to the \ufb01rst term, and the sensory input arising from the second term. Recall that the connectivity\nmatrix Q is essentially the generator matrix of the state transition process, and as such, incorporates\nprior knowledge about the world dynamics. The second term, consisting of the sensory input, con-\ntributes to the state estimator update every time a spike occurs. Thus, a major question relates to\nthe interplay between the a-priori knowledge embedded in the network through Q and the incom-\ning sensory input. In particular, an important question relates to tailoring the system parameters\n(e.g., the tuning curves \u03bbk), to the properties of the external world. As a simple characterization\nof the generator matrix Q, we consider the diagonal and non-diagonal terms. The diagonal term\nqii is related to the average time spent in state i through E[Ti] = \u22121/qii [5], and thus we de\ufb01ne\n\n11 + \u00b7 \u00b7 \u00b7 + q\u22121\n\n\u03c4 (Q) = \u2212(cid:0)q\u22121\n\nN N(cid:1) /N, as a measure of the transition frequency of the process, where\n\nsmall values of \u03c4 correspond to a rapidly changing process. A second relevant measure relates to\nthe regularity in the transition between the states. To quantify this consider a state i, and de\ufb01ne a\nprobability vector qi consisting of the N \u2212 1 elements {Qij}, j 6= i, normalized so that the sum\nof the elements is 1. The entropy of qi is a measure for the state transition irregularity from state i,\nand we de\ufb01ne H(Q) as the average of this entropy over all states. In summary, we lump the main\nproperties of Q into \u03c4 (Q), related to the rapidity of the process, and H(Q), measuring the transition\nregularity. Clearly, these variables are but one heuristic choice for characterizing the Markov pro-\ncess dynamics, but they will enable us to relate the \u2018world dynamics\u2019 to the system behavior. The\nsensory input in\ufb02uence on the system is controlled by the tuning curves. To simplify the analysis we\nassume uniformly placed Gaussian tuning curves, \u03bbk(x) = \u03bbmax exp (\u2212(x \u2212 k\u2206x)2/2\u03c32), which\ncan be characterized by two parameters - the maximum value \u03bbmax and the width \u03c3. Note, however\nthat our model does not require any special constraints on the tuning curves.\n\nFigure 3 examines the system performance under different world setups. We measure the perfor-\nmance using the L1 error of the maximum aposteriori (MAP) estimator built from the posterior\ndistribution generated by the system. The MAP estimator is obtained by selecting the cell with the\nhighest \ufb01ring activity \u03c1i(t), is optimal under the present setting (leading to the minimal probability\nof error), and can be easily implemented in a neural network by a Winner-Take-All circuit. The\nchoice of the L1 error is justi\ufb01ed in this case since the states {si} represent locations on a line,\n\n5\n\n\fthereby endowing the state space with a distance measure. In \ufb01gure 3(a) we can see that as \u03c4 (Q)\nincreases, the error diminishes, an expected result, since slower world dynamics are easier to ana-\nlyze. The effect of H(Q) is opposite - lower entropy implies higher con\ufb01dence in the next state.\nTherefore we can see that the error increases with H(Q) (\ufb01g. 3(b)). The last issue we examine\nrelates to the behavior of the system when an incorrect Q matrix is used (i.e., the world model is\nincorrect). It is clear from \ufb01gure 3(c) that for low values of M (the number of sensory cells), using\nthe wrong Q matrix increases the error level signi\ufb01cantly. However as the value of M increases, the\ndifferences are reduced. This phenomenon is expected, since the more observations are available\nabout the world, the less weight need be assigned to the a-priori knowledge.\n\nr\no\nr\nr\nE\n\n \n\nL\n\n1\n\n1\n\n0.8\n\n0.6\n\n0.4\n\n0.2\n\n1.6\n\n1.4\n\n1.2\n\n1\n\n0.8\n\nr\no\nr\nr\nE\n\n \n\nL\n\n1\n\n0\n0\n\n2\n\n4\n\nt (Q)\n\n6\n\n8\n\n10\n\n1\n\n1.5\n\n2\n\n2.5\nH(Q)\n\n3\n\n3.5\n\n4\n\nr\no\nr\nr\nE\n\n \n\nL\n\n1\n\n10\n\n8\n\n6\n\n4\n\n2\n\n0\n \n0\n\nCorrect model\nWrong Q\u2212matrix, same t (Q)\nWrong Q\u2212matrix, different t (Q)\n\n \n\n100\n\n200\n\nM\n\n300\n\n400\n\n500\n\n(a) Effect of state rapidity\n\n(b) Effect of transition entropy\n\n(c) Effect of misspeci\ufb01cation\n\nFigure 3: State estimation error for different world dynamics and model misspeci\ufb01cation. For (a)\nand (b) M = 17, N = 17, \u03c3 = 3, \u03bbmax = 50, and for (c) N = 25, \u03c3 = 3, \u03bbmax = 50.\n\nIn \ufb01gure 4 we examine the effect of the tuning curve parameters on the system\u2019s performance. Given\na \ufb01xed number of input cells, if the tuning curves are too narrow (\ufb01g. 4(a) top), they will not cover\nthe entire state space. On the other hand, if the tuning curves are too wide (\ufb01g. 4(a) bottom) the cell\u2019s\nresponse is very similar for all states. Therefore we get an error function that has a local minimum\n(\ufb01g. 4(b)). It remains for future work to determine what is the optimal value of \u03c3 for a given model.\nThe effect of different values of \u03bbmax is obvious - higher values of \u03bbmax lead to more spikes per\nsensory cell which increases the system\u2019s accuracy. Clearly, under physiological conditions, where\nhigh \ufb01ring rates are energetically costly, we would expect a tradeoff between accuracy and energy\nexpenditure.\n\nlow s\n\n)\nx\n(\n\nk\n\n40\n\n20\n\n0\n0\n\n)\nx\n(\n\nk\n\n40\n\n20\n\n0\n0\n\n)\nx\n(\n\nk\n\n40\n\n20\n\n0\n0\n\n10\n\n20\n\n30\n\n40\n\n50\n\nx[cm]\n\nmedium s\n\n60\n\n70\n\n80\n\n90\n\n100\n\n10\n\n20\n\n30\n\n40\n\n50\nx[cm]\nhigh s\n\n60\n\n70\n\n80\n\n90\n\n100\n\n10\n\n20\n\n30\n\n40\n\n50\n\nx[cm]\n\n60\n\n70\n\n80\n\n90\n\n100\n\n(a)\n\nr\no\nr\nr\nE\n\n \n\nL\n\n1\n\n14\n\n12\n\n10\n\n8\n\n6\n\n4\n\n2\n\n0\n \n\u22122\n\n \n\n = 50\n\nmax\n\n = 25\n\nmax\n\n = 10\n\nmax\n\n1\n\n2\n\n3\n\n\u22121\n\n0\n\nlog(s )\n(b)\n\nFigure 4: The effect of the tuning curves parameters on performance.\n\n4 Mathematical Framework and Derivations\n\nWe summarize the main mathematical results related to point process \ufb01ltering, adapted mainly from\n[3]. Consider a \ufb01nite-state continuous-time Markov process Xt \u2208 {s1, s2, . . . , sN } with a gener-\nator matrix Q that is being observed via a set of (doubly stochastic) Poisson processes with state-\ndependent rate functions \u03bbk(Xt), k = 1, . . . , M.\nConsider \ufb01rst a single point process observation N t\n0 = {Ns}0\u2264s\u2264t. We denote the joint probability\nlaw for the state and observation process by P1. The objective is to derive a differential equation for\nthe posterior probabilities (1). This is the classic nonlinear \ufb01ltering problem from systems theory\n\n6\n\nl\nl\nl\nl\nl\nl\n\f(e.g. [6]). More generally, the problem can be phrased as computing E1[f (Xt)|N t\ncase of (1), f is a vector function, with components fi(x) = [x = si].\nWe outline the derivation required to obtain such an equation, using a method referred to as\nchange of measure (e.g., [3]). The basic idea is to replace the computationally hard evaluation\n0], by a tractable computation based on a simple probability law. Consider two\nof E1[f (Xt)|N t\nprobability spaces (\u2126, F, P0) and (\u2126, F, P\u221e) that differ only in their probability measures. P1\nis said to be absolutely continuous with respect to P0 (denoted by P1 (cid:28) P0), if for all A \u2208 F,\nP0(A) = 0 \u21d2 P1(A) = 0. Assuming P1 (cid:28) P0, it can be proved that there exists a random variable\nL(\u03c9), \u03c9 \u2208 \u2126, such that for all A \u2208 F,\n\n0], where, in the\n\nP1(A) = E0[1AL] =ZA\n\nL(\u03c9)dP0(\u03c9),\n\n(9)\n\nwhere E0 denotes the expectation with regard to P0. The random variable L is called the Radon-\nNykodim derivative of P1 with respect to P0, and is denoted by L(\u03c9) = dP1(\u03c9)/dP0(\u03c9).\nConsider two continuous-time random processes - Xt,Nt, that have different interpretation under\nthe different probability measures - P0, P1:\n\nNt is a Poisson process with a constant rate of 1, independent of Xt\n\nP0 :(cid:26) Xt is a \ufb01nite-state Markov process, Xt \u2208 {s1, s2, . . . , sN }.\nP1 :(cid:26) Xt is a \ufb01nite-state Markov process, Xt \u2208 {s1, s2, . . . , sN }.\n\nNt is a doubly-stochastic Poisson process with rate function: \u03bb(Xt)\n\n,\n\n.\n\n(10)\n\n(11)\n\nThe following avatar of Bayes\u2019 formula (eq. 3.5 in chap. 6 of [3]), supplies a way to calculate the\nconditional expectation E1[f (Xt)|N t\n\n0] based on P1 in terms of an expectation w.r.t. P0,\n\nE1[f (Xt)|N t\n\n0] =\n\nE0[Ltf (Xt)|N t\n0]\n\nE0[Lt|N t\n0]\n\n,\n\n(12)\n\nwhere Lt = dP1,t/dP0,t, and P0,t and P1,t are the restrictions of P0 and P1, respectively, to the\nsigma-algebra generated by {N t\nUsing (1) and (12) we have\n\n0 }. We refer the reader to [3] for precise de\ufb01nitions.\n\n0, X\u221e\n\npi(t) = E1[fi(Xt)|N t\n\n0] =\n\nE0[Ltfi(Xt)|N t\n0]\n\nE0[Lt|N t\n0]\n\n.\n\n(13)\n\nSince the denominator is independent of i, it can be regarded as a normalization factor. Thus,\nde\ufb01ning \u03c1i(t)\n\n4\n= E0[Ltfi(Xt)|N t\n\nj=1 \u03c1j(t).\n\nBased on the above derivation, one can show ([3], chap. 6.4) that {\u03c1i(t)} obey the stochastic differ-\nential equation (SDE)\n\nd\u03c1i(t) =\n\nN\n\nXj=1\n\nQji\u03c1j(t)dt + (\u03bb(si) \u2212 1)\u03c1i(t)(dNt \u2212 dt).\n\n(14)\n\nA SDE of the form d\u03c1(t) = a(t)dt + b(t)dNt should be interpreted as follows. If at time t, no\njump occurred in the counting process Nt, then \u03c1(t + dt) \u2212 \u03c1(t) \u2248 a(t)dt, where dt denotes an\nin\ufb01nitesimal time interval. If a jump occurred at time t then \u03c1(t + dt) \u2212 \u03c1(t) \u2248 a(t)dt + b(t). Since\nthe jump locations are random, \u03c1(t) is a stochastic process, hence the term SDE.\nNow, this derivation can be generalized to the case where there are M observation processes\nN (1)\nwith different rate functions \u03bb1(Xt), \u03bb2(Xt), . . . , \u03bbM (Xt). In this case the\ndifferential equations for the non-normalized posterior probabilities is\n\n, . . . , N (M )\n\n, N (2)\n\nt\n\nt\n\nt\n\nd\u03c1i(t) =\n\nN\n\nXj=1\n\nQji\u03c1j(t)dt +\n\nM\n\nXk=1\n\n(\u03bbk(si) \u2212 1)\u03c1i(t)(dN (k)\n\nt \u2212 dt)\n\n(15)\n\nRecalling that N (k)\nn is the arrival time of the n-th event in the k-th observation process.\ntk\n\nis a counting process, namely dN (k)\n\nt /dt =Pn \u03b4(t \u2212 tk\n\nt\n\nn), we obtain (2), where\n\n7\n\n0], it follows that pi(t) = \u03c1i(t)/PN\n\n\f5 Discussion\n\nIn this work we have introduced a linear recurrent neural network model capable of exactly imple-\nmenting Bayesian state estimation and prediction from input spike trains in real time. The framework\nis mathematically rigorous and requires few assumptions, is naturally formulated in continuous time,\nand is based directly on spike train inputs, thereby sacri\ufb01cing no temporal resolution. The setup is\nideally suited to the integration of several sensory modalities, and retains its optimality in this setting\nas well. The linearity of the system renders an analytic solution possible, and a full characterization\nin terms of a-priori knowledge and online sensory input. This framework sets the stage for many\npossible extensions and applications, of which we mention several examples. (i) It is important\nto \ufb01nd a natural mapping between the current abstract neural model and more standard biologi-\ncal neural network models. One possible approach was mentioned in Section 2, but other options\nare possible and should be pursued. Additionally, the implementation of the estimation network\n(namely, the variables \u03c1i(t)) using realistic spiking neurons is still open. (ii) At this point the matrix\nQ in (4) is assumed to be known. Combining approaches to learning Q and adapting the tuning\ncurves \u03bbk in real time will lend further plausibility and robustness to the system. (iii) The present\nframework, based on doubly stochastic Poisson processes, can be extended to more general point\nprocesses, using the \ufb01ltering framework described in [10]. (iv) Currently, each world state is repre-\nsented by a single neuron (a grandmother cell). This is clearly a non-robust representation, and it\nwould be worthwhile to develop more distributed and robust representations. Finally, the problem\nof experimental veri\ufb01cation of the framework is a crucial step in future work.\nAcknowledgments The authors are grateful to Rami Atar his helpful advice on nonlinear \ufb01ltering.\n\nReferences\n[1] J.M. Beck and A. Pouget. Exact inferences in a neural implementation of a hidden markov\n\nmodel. Neural Comput, 19(5):1344\u20131361, 2007.\n\n[2] R. Ben-Yishai, R.L. Bar-Or, and H. Sompolinsky. Theory of orientation tuning in visual cortex.\n\nProc Natl Acad Sci U S A, 92(9):3844\u20138, Apr 1995. 542.\n\n[3] P. Br\u00b4emaud. Point Processes and Queues: Martingale Dynamics. Springer, New York, 1981.\n[4] U.T. Eden, L.M. Frank, V. Solo, and E.N. Brown. Dynamic analysis of neural encoding by\n\npoint process adaptive \ufb01ltering. Neural Computation, 16:971\u2013998, 2004.\n\n[5] G.R. Grimmett and D.R. Stirzaker. Probability and Random Processes. Oxford University\n\nPress, third edition, 2001.\n\n[6] A.H. Jazwinsky. Stochastic Processes and Filtering Theory. Academic Press, 1970.\n[7] H.J. Kushner. Dynamical equations for optimal nonlinear \ufb01ltering. J. Differential Equations,\n\n3:179\u2013190, 1967.\n\n[8] R.P.N. Rao. Bayesian computation in recurrent neural circuits. Neural Comput, 16(1):1\u201338,\n\n2004. 825.\n\n[9] R.P.N. Rao. Neural models of Bayesain belief propagation. In K. Doya, S. Ishii, A. Pouget,\n\nand R. P. N. Rao, editors, Bayesian Brain, chapter 11. MIT Press, 2006.\n\n[10] A. Segall, M. Davis, and T. Kailath. Nonlinear \ufb01ltering with counting observations. IEEE\n\nTran. Information Theory,, 21(2):143\u2013149, 1975.\n\n[11] S. Shoham, L.M. Paninski, M.R. Fellows, N.G. Hatsopoulos, J.P. Donoghue, and R.A. Nor-\nman. Statistical encoding model for a primary motor cortical brain-machine interface. IEEE\nTrans Biomed Eng., 52(7):1312\u201322, 2005.\n\n[12] D. L. Snyder. Filtering and detection for doubly stochastic Poisson processes. IEEE Transac-\n\ntions on Information Theory, IT-18:91\u2013102, 1972.\n\n[13] N. Twum-Danso and R. Brockett. Trajectory estimation from place cell data. Neural Netw,\n\n14(6-7):835\u2013844, 2001.\n\n[14] W.M. Wonham. Some applications of stochastic differential equations to optimal nonlinear\n\n\ufb01ltering. J. SIAM Control, 2(3):347\u2013369, 1965.\n\n[15] M. Zakai. On the optimal \ufb01ltering of diffusion processes. Z. Wahrscheinlichkeitheorie verw\n\nGebiete, 11:230\u2013243, 1969.\n\n8\n\n\f", "award": [], "sourceid": 454, "authors": [{"given_name": "Omer", "family_name": "Bobrowski", "institution": null}, {"given_name": "Ron", "family_name": "Meir", "institution": null}, {"given_name": "Shy", "family_name": "Shoham", "institution": null}, {"given_name": "Yonina", "family_name": "Eldar", "institution": null}]}