{"title": "Hidden Markov Model of Cortical Synaptic Plasticity: Derivation of the Learning Rule", "book": "Advances in Neural Information Processing Systems", "page_first": 253, "page_last": 260, "abstract": null, "full_text": "Hidden Markov Model of Cortical Synaptic\nPlasticity: Derivation of the Learning Rule\n\nMichael Eisele\n\nW. M. Keck Center\n\nfor Integrative Neuroscience\n\nSan Francisco, CA 94143-0444\n\nKenneth D. Miller\nW. M. Keck Center\n\nfor Integrative Neuroscience\n\nSan Francisco, CA 94143-0444\n\neisele@phy.ucsf.edu\n\nken@phy.ucsf.edu\n\nAbstract\n\nCortical synaptic plasticity depends on the relative timing of pre- and\npostsynaptic spikes and also on the temporal pattern of presynaptic spikes\nand of postsynaptic spikes. We study the hypothesis that cortical synap-\ntic plasticity does not associate individual spikes, but rather whole \ufb01r-\ning episodes, and depends only on when these episodes start and how\nlong they last, but as little as possible on the timing of individual spikes.\nHere we present the mathematical background for such a study. Stan-\ndard methods from hidden Markov models are used to de\ufb01ne what \u201c\ufb01r-\ning episodes\u201d are. Estimating the probability of being in such an episode\nrequires not only the knowledge of past spikes, but also of future spikes.\nWe show how to construct a causal learning rule, which depends only\non past spikes, but associates pre- and postsynaptic \ufb01ring episodes as if\nit also knew future spikes. We also show that this learning rule agrees\nwith some features of synaptic plasticity in super\ufb01cial layers of rat visual\ncortex (Froemke and Dan, Nature 416:433, 2002).\n\n1 Introduction\n\nCortical synaptic plasticity agrees with the Hebbian learning principle: Neurons that \ufb01re\ntogether, wire together. But many features of cortical plasticity go beyond this simple\nprinciple, such as the dependence on spike-timing or the nonlinear dependence on spike\nfrequency (see [1] or [2] for review). Studying these features may produce a better under-\nstanding of which neurons wire together in the neocortex.\n\nPrevious models of cortical synaptic plasticity [3]-[5] differed in their details, but they\nagreed that nonlinear learning rules are needed to model cortical plasticity. In linear learn-\ning rules, the weight change induced by a presynatic spike would depend only on the post-\nsynaptic spikes, but not on all the other presynaptic spikes. In the cortex, by contrast, the\ncontribution from a presynaptic spike is stronger when it occurs alone than when it oc-\ncurs right after another presynaptic spike [5]. Similar results hold for postsynaptic spikes.\nConsequently, the weight change depends in a complex way on the whole temporal pattern\nof pre- and postsynaptic spikes. Even though this nonlinear dependence can be modeled\nphenomenologically [3]-[5], its biological function remains unknown. We will not pro-\npose such a function here, but reduce this complex dependence to a few principles, whose\n\n\fA\n\npre\n\npost\n\nB\n\nspikes\nfiring\nepisodes\n\nC\n\npre\n\npost\n\nLTP\n\nLTD\n\nLTP or LTD\n\na = 1\n12\n\n2\n\ne (1)> 0\n2\n\nD\n\n1 - a\n\n20\n\n1\n\ne (1) = 1\n1\n\na\n20\n\na\n01\n\n1 - a\n\n01\n\n0\n\ne (1) = 0\n0\n\nLTP\n\nLTD\n\nLTP\n\ntime\n\nFigure 1: A: Usually, models of cortical synaptic plasticity associate pre- and postsynaptic\nspikes directly. They produce long-term potentiation (LTP) when the presynaptic spike\n(pre) precedes the postsynaptic spike (post), and long-term depression (LTD) if the order is\nreversed. When several pre- and postsynaptic spikes are interleaved in time, the outcome\ndepends in a complicated way on the whole spike pattern (LTP or LTD). B: In our model,\npre- and postsynaptic spikes are paired only indirectly. Each spike train is used to estimate\nwhen \ufb01ring episodes start and end. C: These \ufb01ring episodes are then associated, with LTP\nbeing induced if the presynaptic \ufb01ring episode starts before the postsynaptic one and LTD\nif the order is reversed and if the episodes are short. D: Hidden Markov model used to\nestimate when \ufb01ring episodes occur.\n\nfunction may be easier to understand in future studies.\n\n2 Basic learning principle\n\nThe basic principle behind our model is illustrated in \ufb01g. 1. We propose that the learning\nrule does not associate pre- and postsynaptic spikes directly, but rather uses them to esti-\nmate whether the pre- or postsynaptic neuron is currently in a period of rapid \ufb01ring (\u2019\ufb01ring\nepisode\u2019) or a period of little or no \ufb01ring. It then associates the \ufb01ring episodes. When\nthe per- and postsynaptic \ufb01ring episodes overlap, the synapse is strengthened or weakened\ndepending on which one started \ufb01rst, but independent of the precise temporal patterns of\nspikes within a \ufb01ring episode. As a consequence, the contribution of each spike to synaptic\nplasticity will depend on whether it occurs alone, or surrounded by other spikes, and the\nlearning rule will be nonlinear. For the right parameter choice, the nonlinear features of\nthis rule will agree well with nonlinear features of cortical synaptic plasticity.\n\nImplementation of this rule will be done in two steps. Firstly, we will de\ufb01ne what \u201d\ufb01ring\nepisodes\u201d are. Secondly, we will associate the pre- and postsynaptic \ufb01ring episodes. The\n\ufb01rst step uses standard methods from hidden Markov models (see e.g. [6]). The pre- and\npostsynaptic neuron will each be described by a Markov model with three states (\ufb01g. 1D),\nwhich correspond to \ufb01ring episodes (state 2; \ufb01ring probability \u0002\u0001\u0002\u0003\u0005\u0004\u0007\u0006\t\b\u000b\n ), to the silence\nbetween responses (state 0; \ufb01ring probability \r\f\u000e\u0003\u000f\u0004\u0010\u0006\u0012\u0011\u0013\n ), and to the \ufb01rst spike of a new\n\ufb01ring episode (state 1; \ufb01ring probability \u000e\u0014\r\u0003\u000f\u0004\u0010\u0006\u0015\u0011\u0016\u0004 ; duration = 1 time step). As usual,\nthe parameters of the Markov model are the transition probabilities\n, which determine\nhow long \ufb01ring episodes and silent periods are expected to last, and the emission rates\n\u0003\u001d\u001c\u001f\u001e \u0006 , which determine the \ufb01ring rates. \u001c!\u001e\n(\u001c!\u001e#\u0011$\u0004\nat spikes and \u001c\u001f\u001e%\u0011&\n otherwise), \nis the \ufb01ring probability per time step in state\n,\nand \n\u0003\u000f\u0004\u0010\u0006 . In general, the pre- and postsynaptic neuron will have different\nparameters \n\nis the binary observable at time step\n\n\u0003(\n\u000e\u0006)\u0011*\u0004,+-\n\n\u0017\u0019\u0018\u001b\u001a\n\n\u0003\u0005\u0004\u0010\u0006\n\n\u0003\u001d\u001c!\u0006 and\n\n.\n\n\u0018.\u001a\n\n\n\u001a\n\"\n\u001a\n'\n\u001a\n\u001a\n\u001a\n\u0017\n\fOnce the Markov model is de\ufb01ned, one can use standard algorithms (forward and backward\nalgorithm) to estimate, for any given spike sequence, the state probabilities over time. To\nmodel cortical synaptic plasticity, we will increase the synaptic weight whenever the pre-\nand the postsynaptic neuron have simultaneous \ufb01ring episodes (both in state 2), and de-\ncrease the weight whenever the postsynaptic \ufb01ring episode starts \ufb01rst (pre in state 1 while\npost already in state 2):\n\n\u0014\u0016\u0015\u0018\u0017\n\u0015\u001d\u001c\n\n\u0011\u001a\u0019\n\nfor\u0003\u0006\u0005\b\u0007\n\t\nfor\u0003\u0006\u0005\b\u0007\n\t\n\notherwise\n\n\u0003\u0006\u0005\u000e\r\n\u000f\u0011\u0010\n\u0003\u0006\u0005\u000e\r\n\u000f\u0011\u0010\n\n\u0011\u001b\u0019\n\u0011\u001b\u0019\n\n(1)\n\n\u0003\u0004\u0003\u0006\u0005\b\u0007\n\t\n\u001e\f\u000b\n\n\u0003\u0006\u0005\u000e\r\n\u000f\u0011\u0010\n\n\u0002\u0001\nwhere\u0015\u001e\u0017 and\u0015\u001d\u001c\n \"!\n\n(2)\n\n\u0003%$\n\n#\u0001\n\n\u0006'&\b(-\u0003)\u0003\u0006\u0005\b\u0007*\t\n\nare the amplitudes of synaptic potentiation and depression. In general,\nthe states are not known with certainty, only their probabilities are, and the actual weight\nchange is therefore de\ufb01ned as:\n\n\u0005\u000e\r\n\u000f\u0011\u001032\n'10\n\u00033454647+\n\n\u0006-&\u000e(/.\b\u0003\u0006\u0005\u000e\r\n\u000f\u0011\u0010\n\u0011\u001b$,+\u001f\u001c\n\u0005\b\u0007\n\t\nwhere the sum is over all possible pre- and postsynaptic states and(\n45464 . As \ufb01g. 2 shows, this straightforward learning\n\u001c98\n\u0003\u0005\u0004\u0007\u0006?>\n\u0011/;C=-&\u0010\ncombinations:\n\u0006 andB\n\u0005\b\u0007\n\t\ntime step. To \ufb01t the data on spike pairs and triplets [5], we set:\n\u0004\"4ED .)\n;C= , and\u0015\u001d\u001c\n\u0011 96Hz&\n\u0005\b\u0007\n\t\n\u0011 20ms,B\n\n\u001c\u001f\u0006 is the probability\ngiven the whole spike sequence \u001c\nrule produces weight changes that are similar to those seen in cortex [5]. (One can show\nand  only through the two\nthat this particular Markov model depends on the parameters\nis the\n\u0011 34ms,\nThis learning rule is, however, not biologically plausible, because it violates causality. The\nestimates of state probabilities depend not only on past, but also on future observables,\nwhile real synaptic plasticity can depend only on past spikes. To solve this causality prob-\nlem, we will rewrite the learning rule, essentially deriving a new algorithm in place of the\nfamiliar hidden Markov algorithms. We will derive this causal learning rule not only for\nthis speci\ufb01c 3-state model, but for general Markov models.\n\n\u0011<;\"=?>\n\u0003\u000f\u0004\u0010\u0006A@\n\u0005\u000e\r\n\u000f\u0011\u0010\n\u0011 70ms,\u0015\u0018\u0017\n\n\u0006 where;\"=\n\u0005\u000e\r?\u000f\u0011\u0010\n\n\u0011 15ms,:\n\n\u0001\u000f\f\n\n3 General form of the learning rule\n\n3.1 Learning goal\n\n\u0018\u001b\u001a\n\nand the time\n\n\u0003\u0006\u0005\u000e\r\n\u000f\u0011\u0010\n\nand emission probabilities \n\nTo derive the general form of the learning rule for arbitrary pre- and postsynaptic Markov\nmodels, we assume that the transition probabilities\n\u0003\u001d\u001c\u001f\u001e\nare given and that the weight change is some function\n\n\u0002\u0001\n\u0003\u0006\u0005\b\u0007*\t\nof the pre- and postsynaptic states\u0003!\u001e at time\n\u0005\u000e\r\n\u000f\u0011\u0010 were known, the weight\u0001\n\u0005\b\u0007\n\t and\u0003\nnaptic state sequences\u0003\nbe the initial weight\u0001\n\u0003\u0006\u0005\b\u0007*\t\n\u0003\u0006\u0005\u000e\r\n\u000f\u0011\u0010\n\u001e9F\n\u0005\b\u0007*\t?G\n\u0005\b\u0007*\t\n\u000b%J\n\u0005\u000e\r?\u000fK\u0010 . Ideally, we would like to set the weight at time\n\u0005)\u0007*\t and \u001c\nexpectation value of\u0001\n\u001e9F\nG , given the spike trains \u001c\n\u0005\u000e\r\n\u000f\u0011\u0010\n\u0005\b\u0007\n\t and \u001c\n\u0005\b\u0007*\t\n\u0005\b\u0007\n\t\n\u0014 , which we will call \u001cL\u0005\b\u0007\n\t\n, \u001cM\u0005\b\u0007*\t\nthe past values \u001cL\u0005\b\u0007\n\t\n... \u001cL\u0005)\u0007*\t\n, and the present value \u001cL\u0005\b\u0007*\t\n\nIn the current context, the state sequences are unknown and have to be estimated from the\nspike trains \u001c\nequal to the\nthese spike trains are known at time\n\n\u0005\u000e\r\n\u000f\u0011\u0010 . But only part of\n\n\f plus all the previous weight changes:\n\nitself. If the pre- and postsy-\nwould simply\n\nthe synapse has already seen\n. But\n\n. Of the sequence \u001c\n\n\u001e at time\n\n#\u0001\n\nH?I\n\n(4)\n\n(3)\n\n\u001e\n\u0006\n\u0011\n\u0012\n\u0013\n\u001e\n\u000b\n\u001e\n+\n\u001e\n\u0011\n\u0004\n\u000b\n\u001e\n\n\u001f\n\u001a\n\u000b\n'\n\u001e\n\u001e\n\u0011\n0\n\u001c\n\u0014\n\u000b\n\u001c\n\u0001\n\u000b\n\u000b\n\u0017\n\u0003\n\n\u0001\n\u0017\n\u0001\n\f\n+\n\u0017\n\f\n\u0014\n\u0001\n\u0003\n\u0017\n&\n\u0017\n\f\n\u0014\nB\n\u0011\n\u0017\n\u001a\n\u0006\n.\n\u001e\n\u000b\n\u001e\n\u000b\n\"\n2\n\"\n\"\n\"\n\u0001\n\u0003\n\u000b\n\u0003\n\u0011\n\u0001\n\f\n@\n\u001e\n\u001f\n\u0014\n.\nH\n\u000b\nH\n2\n\"\n\u0003\n\u000b\n\u0003\n\"\n\u0014\n\u0001\n\u001e\n\u001c\n\u001c\n\u001e\n\f2/1 triplets; phen. model\n\n2/1 triplets; hidden Markov model\n\n2/1 triplets; linear rule\n\n1\n\n0.5\n\n0\n\nw\nd\n\n\u22120.5\n\n25\n\n5\n\n0\n\u22125\n\u221225\nt2 (ms)\n\n1\n\n0.5\n\n0\n\nw\nd\n\n\u22120.5\n\n25\n\n\u221225\n\n0525\n\n\u22125\nt1 (ms)\n\n5\n\n0\n\u22125\n\u221225\nt2 (ms)\n\n1\n\n0.5\n\n0\n\nw\nd\n\n\u22120.5\n\n25\n\n\u221225\n\n0525\n\n\u22125\nt1 (ms)\n\n5\n\n0\n\u22125\n\u221225\nt2 (ms)\n\n)\ns\nm\n\n(\n \n2\nt\n\n25\n5\n0\n\u22125\n\u221225\n\n\u221225\n\n0525\n\n\u22125\nt1 (ms)\n\nexamples of 2/1 triplets\n\n25 5 0 \u22125 \u221225\n\nt1 (ms)\n\n1/2 triplets; phen. model\n\n1/2 triplets; hidden Markov model\n\n1/2 triplets; linear rule\n\n1\n\n0.5\n\n0\n\nw\nd\n\n\u22120.5\n\n\u221225\n\n\u22125\n\n0\n\n5\n25\nt2 (ms)\n\n\u221225\u22125 0 5 25\n\nt1 (ms)\n\n1\n\n0.5\n\n0\n\nw\nd\n\n\u22120.5\n\n\u221225\n\n\u22125\n\n0\n\n5\n25\nt2 (ms)\n\n1\n\n0.5\n\n0\n\nw\nd\n\n\u22120.5\n\n\u221225\n\n\u22125\n\n0\n\n5\n25\nt2 (ms)\n\n\u221225\u22125 0 5 25\n\nt1 (ms)\n\nexamples of 1/2 triplets\n\n)\ns\nm\n\n(\n \n2\nt\n\n\u221225\n\u22125\n0\n5\n25\n\n\u221225 \u22125 0 5 25\n\nt1 (ms)\n\n\u221225\u22125 0 5 25\n\nt1 (ms)\n\n\u0014 and=\u0005\u0001\n\nit has not yet seen the future sequence \u001c\n\n. All one can\n\u001e accordingly,\nin the future, when the real spike sequence becomes known. Our algorithm\n\nFigure 2: Weight change produced by spike triplets in various models. Our learning rule\n(second column), which depends on the timing of \ufb01ring episodes but only weakly on the\ntiming of individual spikes, and which was implemented using hidden Markov models,\nagrees well with the phenomenological model (\ufb01rst column) that was used in [5, \ufb01g 3b]\nto \ufb01t data from super\ufb01cial layers in rat visual cortex.\nIt certainly agrees better than a\npurely linear rule (third column). Parameters were set so that all three models produce the\nsame results for spike pairs (1 presynaptic and 1 postsynaptic spike). Upper row: Weight\nchange produced by 2 presynaptic and 1 postsynaptic spikes (2/1 triplet). Lower row: 1\npresynaptic and 2 postsynaptic spikes (1/2 triplet).\nand postsynaptic spikes. The small boxes on the right show examples of spike patterns for\n\n=.\u0004 and=3\u0019 are the times between pre-\npositive and negative=\n\u0014 , \u001cM\u0005\b\u0007\n\t\n\u0001 , ..., which we will call \u001cL\u0005\b\u0007*\t\ndo is to make some assumption about what the future spikes will be, set \u0001\nand correct\u0001\n\u001cL\u0005\u000e\r?\u000f\u0011\u0010\n\u001cL\u0005\b\r\n\u000f\u0011\u0010\n\u001cL\u0005\b\r\n\u000f\u0011\u0010\n\u001cM\u0005\b\u0007\n\t\n\u001cM\u0005\b\u0007*\t\n+!\u001cM\u0005\b\u0007*\t\n46454\n\n and \u001cM\u0005\u000e\r\n\u000f\u0011\u0010\n\u0005\b\u0007*\t\n\u001cM\u0005\b\u0007*\t\n\u001cL\u0005\b\u0007\n\t\n\u001cM\u0005\u000e\r\n\u000f\u0011\u0010\n\u001cL\u0005\u000e\r?\u000f\u0011\u0010\n\u001cM\u0005\b\u0007*\t\nand \u001cM\u0005\u000e\r\n\u000f\u0011\u0010\n\u0005\b\u0007\n\t\n, if the timeJ\n\n. The condition that\nwhere \u0001\nall future spikes are 0 is written as \u001c\n\n . One could make other\nassumptions about the future spikes, but all these assumptions would affect only when\nthe weight changes, but not how much it changes in the long run. This is because the\nexpectation value of a past weight change:\n\n\u0001\u0003\u0002\nis the expectation value given the spike sequences \u001c\n\nwill depend little on the future spikes \u001c\ntime\non our assumptions about future spikes.\n\nis much earlier than the\ngrows, most weight changes will lie in the distant past and depend only weakly\n\n. As\n\nassumes no future spikes and sets the weight at time\n\n\u0003\u0006\u0005\u000e\r\n\u000f\u0011\u0010\n\n\u001cL\u0005\b\r\n\u000f\u0011\u0010\n\n\u0003\u0006\u0005\b\u0007*\t\n\nequal to:\n\n\u0005\b\u0007\n\t\n\n\u0005\b\u0007\n\t\n\n\u0005\b\u0007\n\t\n\n#\u0001\n\n(5)\n\n(6)\n\n\u001c\u0006\u0005\n\n\u000b%J\n\nNext we will show how to compute the expectation value in eq. (5) without having to store\nthe past spike trains \u001c\n. To simplify the notation, we will regard each pair of pre- and\n\n\u001e\n\u0017\n\u001e\n\u0017\n\u0017\n\u001e\n\"\n\u0001\n\u001e\n\u0011\n\u0001\n\u001e\nF\n\u0003\n\u000b\n\u0003\nG\n\u0017\n\u0011\n\n\u000b\n\u001e\n\u000b\n\u001c\n\u000b\n\u0017\n\u0011\n\n\u000b\n\u001e\n\u000b\n\u001c\n\u0004\n\u0003\n+\n\u0017\n\u0011\n\u0017\n\u0011\n\u0001\n\u0002\n.\nH\n\u000b\nH\n2\n0\n0\n\u0017\n\u000b\n\u001e\n\u000b\n\u001c\n\u000b\n\u0017\n\u000b\n\u001e\n\u000b\n\u001c\n\u0004\n\u0017\n\u0017\n\"\n\"\n\u001c\n\fpostsynaptic states .\nmodel. We will also combine the pre- and postsynaptic spikes .(\u001cM\u0005\b\u0007\n\t\n\ncan take the two values 0 or 1, to a single observable \u001c\ndesired weight is then equal to:\n\n2 as a state\u0003\n\n\u0003\u0006\u0005\u000e\r\n\u000f\u0011\u0010\n\n\u0003\u0006\u0005\b\u0007*\t\n\nH of a combined pre- and postsynaptic Markov\n\u001cM\u0005\u000e\r\n\u000f\u0011\u0010\n2 , each of which\nH , which can take 4 values. The\n\u0002\u0001\n\u0003\u0004\u0003\nH\"\u000b\u0011J\n\nwith\n\n(7)\n\n+\u001f\u001c\n\n\u0001\u0001\n\nH?I\n\n3.2 Running estimate of state probabilities\n\nTo compute\u0001\n\n\u001e , it is helpful to \ufb01rst compute the probabilities\n\n\u0003\u000e\u0003\u0019\u001e\n\n+\u001f\u001c\n\n\u001c\u0019\u001e\n\n(8)\n\n(9)\n\nof the states given the past and present spikes and assuming that there are no future spikes.\nThe \u0002\n\u0004\u0007\u0006 (this is similar to the familiar\nforward algorithm for hidden Markov models). Write \u0002 as:\n\n\u0006 can be computed recursively, in terms of \u0002\n\n(-\u0003)\u0003\n(-\u0003\u001d\u001c\n\n\u0011\u0004\u0003\n\n\u001c\u001f\u001e\n\n\u0003\u001f\u001e\npresent state\u0003\u001f\u001e , but not on the past state\u0003!\u001e\n(-\u0003\n\u0006'&\n\u0006'&\n\n\u001d+\u0006\u0003\n\n+\u0006\u0003\n\u0006-&\n\n\u0006'&\n\n\u0011\n\t\n\n\u0003\u001d\u001c\u0019\u001e\n\nwith\n\n\u0003\u0019\u001e\n\n+\u001f\u001c\n\n\u0003\u000e\u0003\n\n\u0018.\u001a\n\nBecause of the Markov property, future and present spikes \u001c\nbut not on \u001c\n\n. Thus the enumerator of the last expression is equal to:\n\n\u0014 or on \u001c\n\n\u0006 of having no future spikes after state\n\n\u0011\u0005\u0003\n\n\u001c\u0019\u001e\n\n(10)\n\n\u0006\u0007\u0006'(-\u0003\n\u0017 and \u001c\u0019\u001e depend only on the\n. Similarly,\u0003\u0019\u001e depends only on\u0003\u001f\u001e\n\u00061&\u000e(\n\n\u0011\u0005\u0003\n\n\u0011\b\u0003\n\n(11)\n\n(12)\n(13)\ncan be computed by the\n\n\u0004\u0007\u0006'&\n\n\u0003(\n\u000e\u0006'&\n\n(14)\n\n(-\u0003\u0004\u0003\u0019\u001e\n\n\u001d+\n\n\u0011\b\u0003\n\n\u0003\u0019\u001e\n\nThe probabilities \t\nbackward algorithm:\n\n(-\u0003\n\n\u0011\u001a$,+\n\n&\f\t\n\nThis is a linear equation with constant coef\ufb01cients. As long as the end of the Markov chain\nis far enough in the future, this equation reduces to an eigenvalue problem with the solution\n\n\u0004\u0007\u0006 , where \u000b\n\nis the largest eigenvalue of the matrix with elements \n\n\u0011\u0005\u000b\nis the corresponding eigenvector. As the matrix elements are positive, \u000b will be real,\nand \t\nand the eigenvector will be unique up to a constant factor (except for quite exceptional,\ndisconnected Markov chains, in which it may depend on the choice of end state). The\n\u0006 , which can be expressed in terms of\n\n\u0011\u0005\u0003\n\n\u0003(\n\u000e\u0006\"&\n\nlast unknown factor in eq. (12) is(\n(-\u0003\u0004\u0003\n+,\u0004\u0010\u00061(-\u0003\u001d\u001c\n\n\u0004\u0010\u0006 :\n\u0011\u0004\u0003\n\n\u0003!\u001e\n\n\u0011\b\u0002\n\n\u0011\u0005\u0003\n(15)\nwhere the Markov property was used again. Putting everything together, one gets the update\nrule for \u0002\n\n\u0006 :\n\n\u0006\r\u0006'(-\u0003\n\n\u001d+\u0006\u0003\n\nwith\n\n\u0018.\u001a\n\n\u001c\u001f\u001e\n\n\u0003\u001d\u001c\n\n\u0018.\u001a\n\n\u0003\u001d\u001c\u0019\u001e\n\n(-\u0003\u001d\u001c\n\n\u00061&\u000f\u0002\n\u001c\u0019\u001e \u00061&\n\n\u0003\u001d\u001c\n\n\u00061&\u0010\n\n\u0004\u0010\u0006\n\n&\u000f\t\n\u0006?>\u0011\t\n\u0006\r\u0006'(-\u0003\u001d\u001c\n\n\u0018\u001b\u001a\n\n(16)\n\n\u0004\u0007\u0006\n\n(17)\n(18)\n\nH\n\u000b\nH\nH\n\u000b\nH\n\u0001\n\u001e\n\u0011\n\u0001\n\u001e\nF\n\u0003\nG\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u000b\n\u001c\n\u001c\n\u0005\n\u0001\n\u001e\nF\n\u0003\nG\n\u0011\n\u0001\n\f\n@\n\u001e\n\u001f\n\u0014\n\u0006\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n(\n\u0011\n'\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u001c\n\u001c\n\u0006\n\u001a\n\u0003\n\"\n\u0018\n\u0003\n\"\n+\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\u001f\n\u0018\n\u001e\n\u0011\n'\n\u000b\n\u0003\n\u001e\n\u001c\n\u0014\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u000b\n\u001c\n\u001c\n\u0006\n\u0011\n\u001f\n\u0018\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u0011\n'\n\u000b\n\u001c\n\u0014\n\u000b\n\u001c\n\u001c\n\u001c\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u001c\n\u001c\n\u0006\n\u001c\n\u001c\n\u001c\n\u0014\n\u001c\n\u001c\n\u0017\n\u0011\n\u001e\n\u0011\n'\n(\n\u0003\n\u001c\n\u001e\n\u001e\n\u0011\n'\n(\n\u001e\n\u0011\n'\n+\n\u0003\n\u001e\n\u001c\n\u0014\n\u0003\n\u0003\n\u001e\n\u001c\n\u0014\n\u000b\n\u001c\n\u001c\n\u0006\n\u001a\n\u0003\n\"\n\n\u001a\n\u0017\n&\n\u001c\n\u0014\n\u000b\n\u001c\n\u001c\n\u0006\n\t\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n(\n\u0003\n\u001c\n\u0017\n\u0011\n\u0011\n'\n\u0006\n\u001a\n\u0003\n\"\n'\n\t\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\u001f\n \n\u001c\n\u0017\n\u0011\n\n\u000b\n\u0003\n\u001e\n\u0017\n\u0014\n\u0003\n\u001e\n\u0011\n'\n\u0006\n\u0011\n\u001f\n \n\t\n \n\u0003\n\"\n@\n\n \n\u0017\n\u001a\n \n\t\n\u001a\n\u0003\n\"\n\u0006\n\u001a\n\u0003\n\"\n@\n \n\u0017\n\u001a\n \n\u0003\n\u001c\n\u0014\n\u000b\n\u001c\n\u001c\n\u0002\n\u0018\n\u0003\n\"\n+\n\u001e\n\u001c\n\u0014\n\u000b\n\u001c\n\u001c\n\u0006\n\u0018\n\u0003\n\"\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u0011\n\n\u000b\n\u001c\n\u001c\n\u001c\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u0011\n\u001e\n\u001c\n\u0014\n\u0006\n\u001a\n\u0003\n\"\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\u001f\n\u0018\n\u000e\n\u001e\n\u000b\n\u001c\n\u001c\n\u0018\n\u0003\n\"\n+\n\u000e\n\u0003\n\u000b\n\u001c\n\u001c\n\u0006\n\u0011\n\u0010\n\u000b\n\u001c\n\u001c\n\u001a\n\u0003\n\u0017\n\u001a\n\u0003\n\"\n\u0018\n\u0003\n\"\n+\n\u0010\n\u001e\n\u000b\n\u001c\n\u001c\n\u0006\n\u0011\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u0011\n\n\u000b\n\u001c\n\u001c\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u000b\n\u001c\n\u001c\n\u0006\n\f\u0010)\u0003\n\n\u001c\u0019\u001e\n\n\u0004\u0010\u0006'&\u0010\n\n3.3 Running estimate of weights\n\nUsing the knowledge of the probabilities \u0002\n\n\u001e9F\n\n+!\u001c\n\n\u0001\u0001\n\u0001\u0001\nThe expectation value \u0001\u0001\n\nno pre- or postsynaptic spike at time\nchanges as:\n\n(\u001c\n\n+!\u001c\n+\u001f\u001c\n\nand using\u0002\n&\u000f\t\n\u001c\u0019\u001e \u00061&\n\n\u0018\u001b\u001a\n\n\u0004 :\n\n\u0004\u0007\u0006\u0006\u0005\n\n\u0006?>\u0011\t\n\n#\u0001\n\n\u0006 , one can now compute the weight\n\u001c\u0019\u001e\n\nin this equation will be equal to\u0001\n\n\u0014 , if there is\n\n ). In between spikes, the weight therefore\n(22)\n\n\u001c\u0019\u001e\n\n#\u0001\n\n+\u001f\u001c\n\n\u0006'&\n\n\u001c\u001f\u001e\n\n\u0003\u001f\u001e\n\n(19)\n\n(20)\n\n(21)\n\n(23)\n\n(24)\n\nAt the time of spikes, the weight change is more complex, because earlier weight changes\nhave to be modi\ufb01ed according to the new state information given by the spikes. To compute\nit, let us introduce the quantities\n\n\u0006?>\u0011\t\n\n\u0006\n>\n\n\u000b,&\n\n+\u000b\u0004\u0010\u0006\n\nThe ratio \t\nbut only on\nthe eigenvalue \u000b and the relative size of the elements of the eigenvector \t . If there is no\nis equal\npre- or postsynaptic spike at time\n. In this case, eq. (16) is a linear equation\nto 1, and \u000e\nwith constant coef\ufb01cients, which can be integrated analytically from one spike to the next,\nthereby speeding up the numerical simulation. At pre- or postsynaptic spikes (\u001c\n\n ), \u0010\ncan be computed by summing eq. (16) over\n\n\u0006 does not really depend on\n\u0003\u001d\u001c\n\n ), the normalization factor \u0010\n\n(\u001c\nno longer depends on\n\nor \u001c\n\n\u0018.\u001a\n\n\u00061&\n\n:\n\n\u0011\u0004\u0002\n\nof spikes. Start with:\n\nThe weight is equal to the sum of these\b\nand, as we will see next, the\b\n#\u0001\n\u0014-F\n(-\u0003)\u0003\u0019\u001e\n\u0006'&\n, and it is thus equal to\b\n\n\u001e ,\n, or \u001c\n\u0011\u0004\u0003\n\n(-\u0003)\u0003\n\n#\u0001\n\n\u0001\u0001\n\n\u0006'&\n\n+\u001f\u001c\n\n\u0006 can be computed in a recursive way, even in the presence\n(25)\n\u0001\u0001\n\n\u0003\u001f\u001e\n\n+\u001f\u001c\n\n\u001c\u0019\u001e\n\n\u0005\u0007\u0006\n\n\u0003\u001f\u001e\n\n\u001c\u0019\u001e\n\n\u0006'&\n\u0006-&\n\u001c and \u0003\n\n(26)\n\n, but\n\n\u0011\b\u0003\n\n+!\u001c\n\n\u0011\u0005\u0003\n\nBecause of the Markov property, the last expectation value depends only on \u001c\nnot on \u001c\n\n\u0004\u0010\u0006 . The other two factors\n\n(27)\ncombine to give the same expression that already occurred in equation (9). As shown above\n(eq. (16)), this expression is equal to\n\nwith the same \u000e\n\n\u0018\u001b\u001a\n\n\u0003(\u001c\u0019\u001e\n\n\u0018.\u001a\n\nas before. Putting everything together, one gets the update rule for\b\n\n\u001c\u0019\u001e\n\n\u0004\u0007\u0006\n\n\u0004\u0010\u0006\n\n#\u0001\n\n\u0006A@\n\n\u0018\u001b\u001a\n\n(28)\n\u0006 :\n(29)\n\n\u00061&\t\b\n\n+\u001f\u001c\n\n\u0011\u0004\u0003\n\n\u0004\u0010\u0006\n>\n\u0011\u001b(-\u0003)\u0003\n\n\u00061&\n\n\u001a\n\u0003\n\"\n\u0018\n\u0003\n\"\n\u0011\n\t\n\u001a\n\u0003\n\"\n\u0003\n\t\n\u0018\n\u0003\n\"\n\u0006\n\"\n\"\n\u001e\n\u0011\n\u001e\n\u000b\n\u001c\n\u001c\n\u0006\n\"\n\u001c\n\u001e\n\u0001\n\u0011\n'\n\u001a\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\u000b\n\u001c\n\u001c\n\u0006\n\u0011\n\u0003\n\u0004\n\u001f\n\u001a\n!\n\u0018\n\u0002\n\u0018\n\u0003\n\"\n+\n\u001a\n\u0003\n\u0017\n\u001a\n\u0003\n\"\n\u0018\n\u0003\n\"\n+\n\u0007\n\u001c\n\u0014\n\u001a\n\u0003\n\"\n\u0001\n\u001e\n\u0011\n\u0001\n\u0003\nG\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u001c\n\u001c\n\u0005\n\u0011\n\u0001\n\u001e\n\u001c\n\u0014\nF\n\u0003\nG\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u001c\n\u001c\n\u0005\n@\n\u001f\n\u001a\n\u0003\n'\n\u000b\n\"\n\u0006\n&\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n\u0001\n\u001e\n\u001c\n\u0014\nF\n\u0003\nG\n\u0005\n\u001e\n\u001c\n\"\n\u001e\n\u0011\n\u0001\n\u001e\n\u0011\n\u0001\n\u001e\n\u001c\n\u0014\n@\n\u001f\n\u001a\n\u0003\n'\n\u000b\n\"\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n\b\n\u001a\n\u0003\n\"\n\u0006\n\u001a\n\u0003\n\"\n\u0001\n\n\u0001\n\u001e\nF\n\u0003\nG\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u0011\n'\n\u000b\n\u001c\n\u001c\n\u0005\n\u0001\n\u001e\n\u0011\n\u001f\n\u001a\n\b\n\u001a\n\u0003\n\"\n\u0006\n\u001a\n\u0003\n\"\n\b\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\u0002\n\u001a\n\u0003\n\"\n\u0003\n\u0003\n'\n\u000b\n\"\n\u0006\n@\n\u0001\n\u001e\n\u001c\n\u0003\nG\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u0011\n'\n\u000b\n\u001c\n\u001c\n\u0011\n\u0003\n'\n\u000b\n\"\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n@\n\u001f\n\u0018\n\u001c\n\u0014\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u0011\n'\n\u000b\n\u001c\n\u001c\n\u0002\n\u001a\n\u0003\n\"\n&\n\u0001\n\u001e\n\u001c\n\u0014\nF\n\u0003\nG\n+\n\u001c\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u000b\n\u0003\n\u001e\n\u0011\n'\n\u000b\n\u0003\n\u001e\n\u001c\n\u0014\n\u000b\n\u001c\n\u001c\n\u0005\n'\n\u0017\n\u0018\n\u0003\n\"\n+\n\u0002\n\u0018\n\u0003\n\"\n+\n\u001e\n\u001c\n\u0014\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u000b\n\u0003\n\u001e\n\u0011\n'\n\u000b\n\u001c\n\u001c\n\u0006\n&\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n\u001e\n\u0011\n'\n\u000b\n\u0003\n\u001e\n\u001c\n\u0014\n\u0017\n\u0011\n\n\u000b\n\u001c\n\u001e\n\u000b\n\u001c\n\u001c\n\u0006\n\u000e\n\u000b\n\u001c\n\u001c\n\u0002\n\u0018\n\u0003\n\"\n+\n\u001a\n\u0003\n\"\n\b\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\u0003\n'\n\u000b\n\"\n\u0006\n&\n\u0002\n\u001a\n\u0003\n\"\n\u001f\n\u0018\n\u000e\n\u0003\n\u000b\n\u001c\n\u001c\n\u0018\n\u0003\n\"\n+\n\f\u0001\u0001\n\n+\u001f\u001c\u001f\u001e\n\nTogether with eqs. (16), (17), (19), and (24) this constitutes our learning rule. It is causal,\nbecause it depends only on past, not on future signals, but in the long run it will give the\neq. (16) and the\b\nsame weight change as the standard hidden Markov rule (2). In between spikes, the \u0002\nin\nin eq. (29) evolve according to linear rules, and the weight changes\nthe de\ufb01nition of\u0001\naccording to the simple rule (22). These simpli\ufb01cations are a consequence of assuming, in\n\u001e equal to\u0001\ncould, for example, set\u0001\n\u001e , that there are no future spikes. Other assumptions are possible: One\n\u0005 , assuming that future spikes\nlearning rule for this \u0001\n(not shown), but then the evolution of \u0002 and\b between spikes\noccur with the rate predicted by the Markov model, and one could also derive a causal\nwould be nonlinear and the evolution of\u0001 would also be more complex.\n\u001e as the sum of\u0001\nThis learning rule still has a rather unusual form. Usually, one writes\u0001\nplus some weight change. Our rule can also be written in this form, if the\b are replaced\nby:;\n\u0006'&\n\u001e9F\n\u0005\r\u0006 (31)\n\u0001\u0001\n\u0006 is a measure for how much the weight should be changed if one suddenly learned, with\nInserting the update rule for\b\n\n .\n\n\u0003\u001f\u001e\u0010\u0011\n\u0001\u0001\n. By de\ufb01nition, the;\n\u0006 gives the update rule for;\n\ncertainty, that the neurons are in state\n\n(30)\n\n\u001c\u001f\u001e\n\n\u001c\u0019\u001e\n\n\u001c\u001f\u001e\n\n\u0004\u0007\u0006\n\n#\u0001\n#\u0001\n\u0006'&\ngives the update rule for\u0001\n\u0006'&\n\n#\u0001\n\n\u0018.\u001a\n\n\u0006A@\n\n\u001e :\n\nSumming over\n\n\u0006 :\n\n+\u0005\u001c\nsum to zero:\u0002\n\u0006'&\u000e;\n\u0006'&\n\n\u0004\u0007\u0006\n\n\u0004\u0010\u0006\n\n\u0004\u0010\u0006\n\n\u0018\u001b\u001a\n\n\u0003\u001d\u001c\u0019\u001e\n\n\u0018.\u001a\n\n\u0006 (32)\n(33)\n\n(34)\n\nThe last,; -dependent sum is nonzero only if spikes arrive. It occurs because a new spike\n\nchanges the probability estimates of previous states, and thereby the desired weight.\n\n3.4 Summary of the learning algorithm\n\n\u0005\b\u0007\n\t\n\n\u0011\b\u0002\n\n\u001e but\n\n\u0005\b\u0007\n\t\n\u0006'&\n\nthe weight\u0001\n\u0005\b\u0007*\t\n\n\u0003 denotes the presynaptic and \u0003\npart: \u0002\n\n\u0006 , where\nthe postsynaptic state. However, one needs to update only\n\nTo simplify notation, we combined the pre- and postsynaptic Markov models into a single\none. How does the learning rule look in terms of the original pre- and postsynaptic param-\neters? If the presynaptic model has\ncombined model has\n\n\u0005\u000e\r\n\u000f\u0011\u0010 , then the\n\u0005\b\u0007\n\t states and the postsynaptic one\n\u0005)\u0007*\t\n\u0005\b\r\n\u000f\u0011\u0010 states. At each time step, we have to update not only\n\u0005\u000e\r?\u000fK\u0010 signal traces; , which we will now write as;\u0002\u0001\n\u0005\u000e\r?\u000f\u0011\u0010 of the signal traces \u0002 , because they factorize into a pre- and a postsynaptic\n\u0002\u000e\u0005\u000e\r?\u000fK\u0010\nDe\ufb01ne the weight change#\u0001\n.%\u0003\u0006\u0005\b\u0007\n\t\n\t\u000e\u0005\b\u0007*\t\n\u0005\b\u0007\n\t\n(\u0001\n\n2 for all possible state pairs.\n\u0006'&\n\n\u0006 . The learning algorithm is then given by:\n\u0006 : De\ufb01ne the states and the parameter  and\n\nFind the leading eigenvector \t of both Markov chains in the absence of spikes:\n\nInitialization (\npostsynaptic Markov model.\n\n\u0003\u0006\u0005\b\r\n\u000f\u0011\u0010\n\b\u0005)\u0007*\t\n\nfor arbitrary start state and 0\n\nInitialize \u0001\n\notherwise)\n\n, ; , and \u0002\n\nof the pre- and\n\n\t\u000e\u0005\b\u0007*\t\n\n\f ; ;\n\n\u0005\b\u0007\n\t\n\n\u0011\u0016\n\u0006\u0005\n\n(35)\n\n\u001e\n\u0011\n\u0001\n\u001e\nF\n\u0003\nG\n\u000b\n\u001c\n\u001c\n\u001e\n\u001e\n\u001c\n\u0014\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\b\n\u001a\n\u0003\n\"\n\u0006\n+\n\u0002\n\u001a\n\u0003\n\"\n\u0001\n\u001e\n\u0011\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n\u0003\n\u0001\n\u0003\nG\n+\n\u001c\n\u0017\n\u0011\n\n\u000b\n\u000b\n'\n\u000b\n\u001c\n\u001c\n\u0005\n+\n\u0001\n\u001e\nF\n\u0003\nG\n\u0017\n\u0011\n\n\u000b\n\u000b\n\u001c\n\u001c\n;\n\u001a\n\u0003\n\"\n'\n\u001a\n;\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\u001a\n\u0003\n\"\n\u001a\n\u0003\n\"\n;\n\u001a\n\u0003\n\"\n\u0006\n\u0011\n\u0003\n\u0003\n'\n\u000b\n\"\n\u0006\n+\n\u0001\n\u001e\n\u0006\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n@\n\u001f\n\u0018\n\u000e\n\u0003\n\u000b\n\u001c\n\u001c\n\u0006\n\u0003\n;\n\u0018\n\u0003\n\"\n+\n@\n\u0002\n\u0018\n\u0003\n\"\n+\n\u0001\n\u001e\n\u001c\n\u0014\n\u0011\n\u0003\n\u0003\n'\n\u000b\n\"\n\u0006\n+\n\u0001\n\u001e\n@\n\u0001\n\u001e\n\u001c\n\u0014\n\u0002\n\u001a\n\u0003\n\"\n\u001f\n\u0018\n\u000e\n\u0003\n\u001c\n\u001e\n\u000b\n\u001c\n\u001c\n\u0018\n\u0003\n\"\n+\n'\n\u0001\n\u001e\n\u0011\n\u0001\n\u001e\n\u001c\n\u0014\n@\n\u001f\n\u001a\n\u0003\n'\n\u000b\n\"\n\u0002\n\u001a\n\u0003\n\"\n\u0006\n@\n\u001f\n\u0018\n!\n\u001a\n\u000e\n\u000b\n\u001c\n\u001c\n;\n\u0018\n\u0003\n\"\n+\n\n\n\n&\n\n\n&\n\n\u0018\n\u0003\n\"\n\n@\n\n\u0001\n\u0018\n\u0003\n\"\n\u0006\n\u0001\n\u0003\n\"\n\u0018\n\u0003\n\"\n\u0004\n\"\n\u0011\n\n\u0017\n\u001e\n\u000b\n\u001e\n\u000b\n\"\n\u000b\n&\n\u001a\n\u0011\n\u001f\n\u0018\n\u0018\n\u0003\n\n\u0017\n\u001a\n\u0018\n&\n\u0018\n\u0011\n\u0001\n\u0002\n\u0011\n\u0004\n\f\u0004 Recursion \u0003\n\n\u0005\b\u0007*\t\n\u000e,\u0005\b\u0007*\t\n\u0005\b\u0007*\t\n\n\u0018\u001b\u001a\n\n\u0018\u001b\u001a\nand analogous equations for \u0010\n\n\u0006 :\n\n45464\n\u0005\b\u0007*\t\n\n\u0018\u001b\u001a\n\n\u0002\u000e\u0005\b\u0007*\t\n\b\u0005\b\u0007\n\t\n\u0005\b\u0007\n\t\n\n \"!\n#\u0001\n\n\u0002\u0001\n\u0003%$\n\n\b\u0005)\u0007*\t\n\u001cM\u0005\b\u0007\n\t\n\u0005\b\u0007*\t\n\u0005\b\r\n\u000f\u0011\u0010\n\u0002\u000e\u0005\b\u0007\n\t\n\u0006'&\n\u00061&\n\n\t\u000e\u0005\b\u0007*\t\n\u0005\b\u0007\n\t\n\n\u0018\u001b\u001a\n\n\u0018\u001b\u001a\n\n\u0005\b\u0007\n\t\n\u0003\u001d\u001cL\u0005\b\u0007\n\t\n\u0006'&\n\t\u000e\u0005\b\u0007*\t\n\u0005\b\u0007\n\t\n\u00061&\n\u0005\u000e\r\n\u000f\u0011\u0010 .\n\u0005\b\r\n\u000f\u0011\u0010 , and \u0002\n \"!\n\u0002\u000e\u0005\u000e\r\n\u000f\u0011\u0010\n\u0005\u000e\r?\u000f\u0011\u0010\n\u0005\b\u0007*\t\n\u0005\b\u0007\n\t and \u001c\n\n\u0005\b\u0007\n\t\n&\u000f\t\u000e\u0005)\u0007*\t\n\n\t\u000e\u0005\b\u0007*\t\n\n\u0006\u0002\u0001\n\n\u000e,\u0005\u000e\r\n\u000f\u0011\u0010\n\u0005\u000e\r\n\u000f\u0011\u0010\n\n\u0018\u001b\u001a\n\n\u0018\u001b\u001a\n\n\u0005\b\u0007\n\t\n\u0005\b\u0007\n\t\n\u0005\u000e\r\n\u000f\u0011\u0010 .\n\n(36)\n\n(37)\n\n(38)\n\n(39)\n\n(40)\n\n(41)\n\n&\u000e;\n\n\u0004 Terminate at the end of the spike sequences \u001c\n\n4 Conclusion\n\nThis demonstrates that the basic principle of associating not individual spikes, but whole\n\ufb01ring episodes, can be implemented in a causal learning rule, which depends only on past\nsignals. This rule does not have to store the time of all past spikes, but only a few signal\n\ntraces \u0002 and ; , and may thus be biologically plausible. For the right parameter choice,\n\nit agrees well with some nonlinear features of cortical synaptic plasticity (\ufb01g. 2). This\ndoes not imply that actual synaptic plasticity follows the same rule, but only that these\nparticular features are consistent with our basic principle. Based on the predictions of\nthis rule, one could design more precise experimental tests of whether cortical synaptic\nplasticity associates individual spikes or whole \ufb01ring episodes.\n\nAcknowledgments\n\nThis work was supported by R01-EY11001. We thank T. Sejnowski for his comments on a\nsimilar type of learning rules, which he suggested to call \u201dhidden Hebbian learning\u201d. The\nsecond author (KM) would like to emphasize that his contribution to this paper was limited\nto assistance in writing.\n\nReferences\n\n[1] G.-Q. Bi and M.-M. Poo Synaptic modi\ufb01cation by correlated activity: Hebb\u2019s postulate revisited.\n\nAnn. Rev. Neurosci., 24:139\u2013166, 2001.\n\n[2] O. Paulsen and T. J. Sejnowski. Natural patterns of activity and long-term synaptic plasticity.\n\nCurr Opin Neurobiol., 10:172\u2013179, 2000.\n\n[3] W. Senn, H. Markram, and M. Tsodyks. An algorithm for modifying neurotransmitter release\n\nprobability based on pre- and postsynaptic spike timing. Neural Comput., 13:35\u201367, 2001.\n\n[4] P. J. Sjostrom, Turrigiano G. G., and S. B. Nelson. Rate, timing, and cooperativity jointly deter-\n\nmine cortical synaptic plasticity. Neuron, 32:1149\u20131164, 2001.\n\n[5] R. C. Froemke and Y. Dan. Spike-timing-dependent synaptic modi\ufb01cation induced by natural\n\nspike trains. Nature, 416:433\u2013438, 2002.\n\n[6] L. R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recogni-\n\ntion. Proceedings of the IEEE, 77:257\u2013286, 1989.\n\n\"\n\u0011\n\u0004\n\u000b\n\u0019\n\u000b\n\u0010\n\u0011\n\n\u001f\n\u0018\n&\n\u001a\n\u001e\n\u0017\n&\n\u001a\n>\n\u0003\n\u000b\n&\n\u0018\n\u001c\n\u0014\n\u0011\n\u0010\n&\n\u001a\n\u0003\n\u001e\n\u0017\n&\n\u001a\n>\n\u0003\n\u000b\n\u0018\n\u0006\n\u0002\n\u001a\n\u0003\n\u001f\n\u0018\n\u000e\n&\n\u0002\n\u0018\n\u000b\n\u000e\n;\n\u0001\n\u0011\n\u001f\n\u001a\n\u0003\n$\n\u000b\n'\n\u000b\n\"\n \n&\n\u001a\n@\n\u001f\n\u001a\n\u001f\n\u0001\n!\n\u0018\n\u000e\n\u0001\n \n&\n\u0001\n\u0018\n;\n \n\u001a\n\u0003\n\u0003\n\u000b\n'\n\u000b\n\"\n\u0006\n+\n;\n\u0001\n\u0002\n \n&\n\u0002\n\u001a\n@\n\u001f\n\u0001\n!\n\u0018\n\u000e\n\u0001\n \n&\n\u000e\n&\n;\n\u0001\n\u0018\n\u0001\n\u0003\n\u0001\n@\n;\n\u0001\n\f", "award": [], "sourceid": 2254, "authors": [{"given_name": "Michael", "family_name": "Eisele", "institution": null}, {"given_name": "Kenneth", "family_name": "Miller", "institution": null}]}