{"title": "A framework for studying synaptic plasticity with neural spike train data", "book": "Advances in Neural Information Processing Systems", "page_first": 2330, "page_last": 2338, "abstract": "Learning and memory in the brain are implemented by complex, time-varying changes in neural circuitry. The computational rules according to which synaptic weights change over time are the subject of much research, and are not precisely understood. Until recently, limitations in experimental methods have made it challenging to test hypotheses about synaptic plasticity on a large scale. However, as such data become available and these barriers are lifted, it becomes necessary to develop analysis techniques to validate plasticity models. Here, we present a highly extensible framework for modeling arbitrary synaptic plasticity rules on spike train data in populations of interconnected neurons. We treat synaptic weights as a (potentially nonlinear) dynamical system embedded in a fully-Bayesian generalized linear model (GLM). In addition, we provide an algorithm for inferring synaptic weight trajectories alongside the parameters of the GLM and of the learning rules. Using this method, we perform model comparison of two proposed variants of the well-known spike-timing-dependent plasticity (STDP) rule, where nonlinear effects play a substantial role. On synthetic data generated from the biophysical simulator NEURON, we show that we can recover the weight trajectories, the pattern of connectivity, and the underlying learning rules.", "full_text": "A framework for studying synaptic plasticity\n\nwith neural spike train data\n\nScott W. Linderman\nHarvard University\n\nCambridge, MA 02138\n\nChristopher H. Stock\n\nHarvard College\n\nCambridge, MA 02138\n\nRyan P. Adams\nHarvard University\n\nCambridge, MA 02138\n\nswl@seas.harvard.edu\n\ncstock@post.harvard.edu\n\nrpa@seas.harvard.edu\n\nAbstract\n\nLearning and memory in the brain are implemented by complex, time-varying\nchanges in neural circuitry. The computational rules according to which synaptic\nweights change over time are the subject of much research, and are not precisely\nunderstood. Until recently, limitations in experimental methods have made it chal-\nlenging to test hypotheses about synaptic plasticity on a large scale. However, as\nsuch data become available and these barriers are lifted, it becomes necessary\nto develop analysis techniques to validate plasticity models. Here, we present\na highly extensible framework for modeling arbitrary synaptic plasticity rules\non spike train data in populations of interconnected neurons. We treat synap-\ntic weights as a (potentially nonlinear) dynamical system embedded in a fully-\nBayesian generalized linear model (GLM). In addition, we provide an algorithm\nfor inferring synaptic weight trajectories alongside the parameters of the GLM and\nof the learning rules. Using this method, we perform model comparison of two\nproposed variants of the well-known spike-timing-dependent plasticity (STDP)\nrule, where nonlinear effects play a substantial role. On synthetic data generated\nfrom the biophysical simulator NEURON, we show that we can recover the weight\ntrajectories, the pattern of connectivity, and the underlying learning rules.\n\nIntroduction\n\n1\nSynaptic plasticity is believed to be the fundamental building block of learning and memory in the\nbrain. Its study is of crucial importance to understanding the activity and function of neural circuits.\nWith innovations in neural recording technology providing access to the simultaneous activity of\nincreasingly large populations of neurons, statistical models are promising tools for formulating and\ntesting hypotheses about the dynamics of synaptic connectivity. Advances in optical techniques [1,\n2], for example, have made it possible to simultaneously record from and stimulate large populations\nof synaptically connected neurons. Armed with statistical tools capable of inferring time-varying\nsynaptic connectivity, neuroscientists could test competing models of synaptic plasticity, discover\nnew learning rules at the monosynaptic and network level, investigate the effects of disease on\nsynaptic plasticity, and potentially design stimuli to modify neural networks.\nDespite the popularity of GLMs for spike data, relatively little work has attempted to model the\ntime-varying nature of neural interactions. Here we model interaction weights as a dynamical system\ngoverned by parametric synaptic plasticity rules. To perform inference in this model, we use particle\nMarkov Chain Monte Carlo (pMCMC) [3], a recently developed inference technique for complex\ntime series. We use this new modeling framework to examine the problem of using recorded data to\ndistinguish between proposed variants of spike-timing-dependent plasticity (STDP) learning rules.\n\n1\n\n\fFigure 1: A simple network of four sparsely connected neurons whose synaptic weights are changing over time.\nHere, the neurons have inhibitory self connections to mimic refractory effects, and are connected via a chain of\nexcitatory synapses, as indicated by the nonzero entries A1\u21922, A2\u21923, and A3\u21924. The corresponding weights\nof these synapses are strengthening over time (darker entries in W ), leading to larger impulse responses in the\n\ufb01ring rates and a greater number of induced post-synaptic spikes (black dots), as shown below.\n\n2 Related Work\nThe GLM is a probabilistic model that considers spike trains to be realizations from a point process\nwith conditional rate \u03bb(t) [4, 5]. From a biophysical perspective, we interpret this rate as a nonlinear\nfunction of the cell\u2019s membrane potential. When the membrane potential exceeds the spiking thresh-\nold potential of the cell, \u03bb(t) rises to re\ufb02ect the rate of the cell\u2019s spiking, and when the membrane\npotential decreases below the spiking threshold, \u03bb(t) decays to zero. The membrane potential is\nmodeled as the sum of three terms: a linear function of the stimulus, I(t), for example a low-pass\n\ufb01ltered input current, the sum of excitatory and inhibitory PSPs induced by presynaptic neurons, and\na constant background rate. In a network of N neurons, let Sn = {sn,m}Mn\nm=1 \u2282 [0, T ] be the set of\nobserved spike times for neuron n, where T is the duration of the recording and Mn is the number\nof spikes. The conditional \ufb01ring rate of a neuron n can be written,\n\n\uf8eb\uf8edbn +\n\n(cid:90) t\n\n0\n\n\u03bbn(t) = g\n\nkn(t \u2212 \u03c4 ) \u00b7 I(\u03c4 ) d\u03c4 +\n\nhn(cid:48)\u2192n(t \u2212 sn(cid:48),m) \u00b7 I[sn(cid:48),m < t]\n\n(1)\n\n\uf8f6\uf8f8 ,\n\nN(cid:88)\n\nMn(cid:48)(cid:88)\n\nn(cid:48)=1\n\nm=1\n\nwhere bn is the background rate, the second term is a convolution of the (potentially vector-valued)\nstimulus with a linear stimulus \ufb01lter, kn(\u2206t), and the third is a linear summation of impulse re-\nsponses, hn(cid:48)\u2192n(\u2206t), which preceding spikes on neuron n(cid:48) induce on the membrane potential of\nneuron n. Finally, the rectifying nonlinearity g : R \u2192 R+ converts this linear function of stimulus\nand spike history into a nonnegative rate. While the spiking threshold potential is not explicitly\nmodeled in this framework, it is implicitly inferred in the amplitude of the impulse responses.\nFrom this semi-biophysical perspective it is clear that one shortcoming of the standard GLM is that it\ndoes not account for time-varying connectivity, despite decades of research showing that changes in\nsynaptic weight occur over a variety of time scales and are the basis of many fundamental cognitive\nprocesses. This absence is due, in part, to the fact that this direct biophysical interpretation is not\nwarranted in most traditional experimental regimes, e.g., in multi-electrode array (MEA) recordings\nwhere electrodes are relatively far apart. However, as high resolution optical recordings grow in\npopularity, this assumption must be revisited; this is a central motivation for the present model.\nThere have been a few efforts to incorporate dynamics into the GLM. Stevenson and Koerding [6]\nextended the GLM to take inter-spike intervals as a covariates and formulated a generalized bilinear\nmodel for weights. Eldawlatly et al. [7] modeled the time-varying parameters of a GLM using a\ndynamic Bayesian network (DBN). However, neither of these approaches accommodate the breadth\nof synaptic plasticity rules present in the literature. For example, parametric STDP models with hard\n\n2\n\ntime\fbounds on the synaptic weight are not congruent with the convex optimization techniques used by\n[6], nor are they naturally expressed in a DBN. Here we model time-varying synaptic weights as a\npotentially nonlinear dynamical system and perform inference using particle MCMC.\nNonstationary, or time-varying, models of synaptic weights have also been studied outside the con-\ntext of GLMs. For example, Petreska et al. [8] applied hidden switching linear dynamical sys-\ntems models to neural recordings. This approach has many merits, especially in traditional MEA\nrecordings where synaptic connections are less likely and nonlinear dynamics are not necessarily\nwarranted. Outside the realm of computational neuroscience and spike train analysis, there exist a\nnumber of dynamic statistical models, such as West et al. [9], which explored dynamic generalized\nlinear models. However, the types of models we are interested in for studying synaptic plasticity\nare characterized by domain-speci\ufb01c transition models and sparsity structure, and until recently, the\ntools for effectively performing inference in these models have been limited.\n3 A Sparse Time-Varying Generalized Linear Model\nIn order to capture the time-varying nature of synaptic weights, we extend the standard GLM by \ufb01rst\nfactoring the impulse responses in the \ufb01ring rate of Equation 1 into a product of three terms:\n\nhn(cid:48)\u2192n(\u2206t, t) \u2261 An(cid:48)\u2192n Wn(cid:48)\u2192n(t) rn(cid:48)\u2192n(\u2206t).\n\ni.e. (cid:82) \u221e\n\n(2)\nHere, An(cid:48)\u2192n \u2208 {0, 1} is a binary random variable indicating the presence of a direct synapse\nfrom neuron n(cid:48) to neuron n, Wn(cid:48)\u2192n(t) : [0, T ] \u2192 R is a non stationary synaptic \u201cweight\u201d tra-\njectory associated with the synapse, and rn(cid:48)\u2192n(\u2206t) is a nonnegative, normalized impulse response,\n0 rn(cid:48)\u2192n(\u03c4 )d\u03c4 = 1. Requiring rn(cid:48)\u2192n(\u2206t) to be normalized gives meaning to the synaptic\nweights: otherwise W would only be de\ufb01ned up to a scaling factor. For simplicity, we assume r(\u2206t)\ndoes not change over time, that is, only the amplitude and not the duration of the PSPs are time-\nvarying. This restriction could be adapted in future work.\nAs is often done in GLMs, we model the normalized impulse responses as a linear combination of\nbasis functions. In order to enforce the normalization of r(\u00b7), however, we use a convex combination\nof normalized, nonnegative basis functions. That is,\n\nrn(cid:48)\u2192n(\u2206t) \u2261 B(cid:88)\n\n\u03b2(n(cid:48)\u2192n)\n\nb\n\nrb(\u2206t),\n\nwhere(cid:82) \u221e\n\n0 rb(\u03c4 ) d\u03c4 = 1, \u2200b and(cid:80)B\n\nb=1\n\nb=1 \u03b2(n(cid:48)\u2192n)\n\nb\n\n= 1, \u2200n, n(cid:48). The same approach is used to model\n\nthe stimulus \ufb01lters, kn(\u2206t), but without the normalization and non-negativity constraints.\nThe binary random variables An(cid:48)\u2192n, which can be collected into an N \u00d7 N binary matrix A,\nmodel the connectivity of the synaptic network. Similarly, the collection of weight trajecto-\nries {{Wn(cid:48)\u2192n(t)}}n(cid:48),n, which we will collectively refer to as W (t), model the time-varying synap-\ntic weights. This factorization is often called a spike-and-slab prior [10], and it allows us to separate\nour prior beliefs about the structure of the synaptic network from those about the evolution of synap-\ntic weights. For example, in the most general case we might leverage a variety of random network\nmodels [11] as prior distributions for A, but here we limit ourselves to the simplest network model,\nthe Erd\u02ddos-Renyi model. Under this model, each An(cid:48)\u2192n is an independent identically distributed\nBernoulli random variable with sparsity parameter \u03c1.\nFigure 1 illustrates how the adjacency matrix and the time-varying weights are integrated into the\nGLM. Here, a four-neuron network is connected via a chain of excitatory synapses, and the synapses\nstrengthen over time due to an STDP rule. This is evidenced by the increasing amplitude of the\nimpulse responses in the \ufb01ring rates. With larger synaptic weights comes an increased probability\nof postsynaptic spikes, shown as black dots in the \ufb01gure. In order to model the dynamics of the\ntime-varying synaptic weights, we turn to a rich literature on synaptic plasticity and learning rules.\n3.1 Learning rules for time-varying synaptic weights\nDecades of research on synapses and learning rules have yielded a plethora of models for the evolu-\ntion of synaptic weights [12]. In most cases, this evolution can be written as a dynamical system,\n\ndW (t)\n\ndt\n\n= (cid:96) (W (t), {sn,m : sn,m < t} ) + \u0001(W (t), t),\n\n3\n\n\ffunctions.\n\nwhere (cid:96) is a potentially nonlinear learning rule that determines how synaptic weights change as a\nfunction of previous spiking. This framework encompasses rate-based rules such as the Oja rule\n[13] and timing-based rules such as STDP and its variants. The additive noise, \u0001(W (t), t), need not\nbe Gaussian, and many models require truncated noise distributions.\nintuition, many common learning rules factor into a product of sim-\nFollowing biological\npler\nFor example, STDP (de\ufb01ned below) updates each synapse indepen-\ndently such that dWn(cid:48)\u2192n(t)/dt only depends on Wn(cid:48)\u2192n(t) and the presynaptic spike his-\ntory Sn<t = {sn,m : sn,m < t}. Biologically speaking, this means that plasticity is local to the\nsynapse. More sophisticated rules allow dependencies among the columns of W . For example, the\nincoming weights to neuron n may depend upon one another through normalization, as in the Oja\nrule [13], which scales synapse strength according to the total strength of incoming synapses.\nExtensive research in the last \ufb01fteen years has identi\ufb01ed the relative spike timing between the pre-\nand postsynaptic neurons as a key component of synaptic plasticity, among other factors such as\nmean \ufb01ring rate and dendritic depolarization [14]. STDP is therefore one of the most prominent\nlearning rules in the literature today, with a number of proposed variants based on cell type and\nbiological plausibility. In the experiments to follow, we will make use of two of these proposed vari-\nants. First, consider the canonical STDP rule with a \u201cdouble-exponential\u201d function parameterized\nby \u03c4\u2212, \u03c4+, A\u2212, and A+ [15], in which the effect of a given pair of pre-synaptic and post-synaptic\nspikes on a weight may be written:\n\n(cid:96) (Wn(cid:48)\u2192n(t),Sn(cid:48),Sn) = I[t \u2208 Sn] (cid:96)+(Sn(cid:48); A+, \u03c4+) \u2212 I[t \u2208 Sn(cid:48)] (cid:96)\u2212(Sn; A\u2212, \u03c4\u2212),\n\n(3)\n\n(cid:96)+(Sn(cid:48); A+, \u03c4+) =\n\nA+ e(t\u2212sn(cid:48) ,m)/\u03c4+\n\n(cid:96)\u2212(Sn; A\u2212, \u03c4\u2212) =\n\nA\u2212 e(t\u2212sn,m)/\u03c4\u2212 .\n\n(cid:88)\n\nsn,m\u2208Sn<t\n\n(cid:88)\n\nsn(cid:48) ,m\u2208Sn(cid:48)<t\n\nThis rule states that weight changes only occur at the time of pre- or post-synaptic spikes, and that\nthe magnitude of the change is a nonlinear function of interspike intervals.\nA slightly more complicated model known as the multiplicative STDP rule extends this by bounding\nthe weights above and below by Wmax and Wmin, respectively [16]. Then, the magnitude of the\nweight update is scaled by the distance from the threshold:\n\n(cid:96) (Wn(cid:48)\u2192n(t),Sn(cid:48),Sn) = I[t \u2208 Sn] \u02dc(cid:96)+(Sn(cid:48); A+, \u03c4+) (Wmax \u2212 Wn(cid:48)\u2192n(t)),\n\u2212 I[t \u2208 Sn(cid:48)] \u02dc(cid:96)\u2212(Sn; A\u2212, \u03c4\u2212) (Wn(cid:48)\u2192n(t) \u2212 Wmin).\n\nthe synaptic weights always\n\nHere, by setting \u02dc(cid:96)\u00b1 = min((cid:96)\u00b1, 1), we enforce that\nwithin [Wmin, Wmax]. With this rule, it often makes sense to set Wmin to zero.\nSimilarly, we can construct an additive, bounded model which is identical to the standard additive\nSTDP model except that weights are thresholded at a minimum and maximum value. In this model,\nthe weight never exceeds its set lower and upper bounds, but unlike the multiplicative STDP rule,\nthe proposed weight update is independent of the current weight except at the boundaries. Likewise,\nwhereas with the canonical STDP model it is sensible to use Gaussian noise for \u0001(t) in the bounded\nmultiplicative model we use truncated Gaussian noise to respect the hard upper and lower bounds\non the weights. Note that this noise is dependent upon the current weight, Wn(cid:48)\u2192n(t).\nThe nonlinear nature of this rule, which arises from the multiplicative interactions among the pa-\nrameters, \u03b8(cid:96) = {A+, \u03c4+, A\u2212, \u03c4\u2212, Wmax, Wmax}, combined with the potentially non-Gaussian noise\nmodels, pose substantial challenges for inference. However, the computational cost of these detailed\nmodels is counterbalanced by dramatic expansions in the \ufb02exibility of the model and the incorpora-\ntion of a priori knowledge of synaptic plasticity. These learning models can be interpreted as strong\nregularizers of models that would otherwise be highly underdetermined, as there are N 2 weight tra-\njectories and only N spike trains. In the next section we will leverage powerful new techniques for\nBayesian inference in order to capitalize on these expressive models of synaptic plasticity.\n4\nThe traditional approach to inference in the standard GLM is penalized maximum likelihood esti-\nmation. The log likelihood of a single conditional Poisson process is well known to be,\n\nInference via particle MCMC\n\nL(cid:0)\u03bbn(t); {Sn}N\n\nn=1, I(t)(cid:1) = \u2212\n\n(cid:90) T\n\nMn(cid:88)\n\n(4)\nfall\n\n\u03bbn(t) dt +\n\nlog (\u03bbn(sn,m)) ,\n\n(5)\n\nm=1\n\n0\n\n4\n\n\ft }P\n\nticle weights1, \u03c9p, which approximate the true distribution via Pr(W t) \u2248(cid:80)P\n\nand the log likelihood of a population of non-interacting spike trains is simply the sum of\neach of the log likelihoods for each neuron.\nThe likelihood depends upon the parame-\nters \u03b8GLM = {bn, kn,{hn(cid:48)\u2192n(\u2206t)}N\nn(cid:48)=1} through the de\ufb01nition of the rate function given in Equa-\ntion 1. For some link functions g, the log likelihood is a concave function of \u03b8GLM, and the MLE can\nbe found using ef\ufb01cient optimization techniques. Certain dynamical models, namely linear Gaus-\nsian latent state space models, also support ef\ufb01cient inference via point process \ufb01ltering techniques\n[17].\nDue to the potentially nonlinear and non-Gaussian nature of STDP, these existing techniques are\nnot applicable here. Instead we use particle MCMC [3], a powerful technique for inference in time\nseries. Particle MCMC samples the posterior distribution over weight trajectories, W (t), the adja-\ncency matrix A, and the model parameters \u03b8GLM and \u03b8(cid:96), given the observed spike trains, by combin-\ning particle \ufb01ltering with MCMC. We represent the conditional distribution over weight trajectories\nwith a set of discrete particles. Let the instantaneous weights at (discretized) time t be represented\nby a set of P particles, {W (p)\np=1. The particles live in RN\u00d7N and are assigned normalized par-\n(W t).\nParticle \ufb01ltering is a method of inferring a distribution over weight trajectories by iteratively propa-\ngating forward in time and reweighting according to how well the new samples explain the data. For\neach particle W (p)\nat time t, we propagate forward one time step using the learning rule to obtain\na particle W (p)\nt+1. Then, using Equation 5, we evaluate the log likelihood of the spikes that occurred\nin the window [t, t + 1) and update the weights. Since some of these particles may have very low\nweights, after each step we resample the particles. After the T -th time step we are left with a set of\nweight trajectories {(W (p)\nParticle \ufb01ltering only yields a distribution over weight trajectories, and implicitly assumes that the\nother parameters have been speci\ufb01ed. Particle MCMC provides a broader inference algorithm for\nboth weights and other parameters. The idea is to interleave conditional particle \ufb01ltering steps\nthat sample the weight trajectory given the current model parameters and the previously sampled\nweights, with traditional Gibbs updates to sample the model parameters given the current weight\ntrajectory. This combination leaves the stationary distribution of the Markov chain invariant and\nallows joint inference over weights and parameters. Gibbs updates for the remaining model param-\neters, including those of the learning rule, are described in the supplementary material.\nCollapsed sampling of A and W (t)\nIn addition to sampling of weight trajectories and model\nparameters, particle MCMC approximates the marginal likelihood of entries in the adjacency ma-\ntrix, A, integrating out the corresponding weight trajectory. We have, up to a constant,\nPr(An(cid:48)\u2192n | S, \u03b8(cid:96), \u03b8GLM, A\u00acn(cid:48)\u2192n, W \u00acn(cid:48)\u2192n(t))\n\np=1, each associated with a particle weight \u03c9p.\n\n0 , . . . , W (p)\n\nT )}P\n\np=1 \u03c9p \u03b4W (p)\n\nt\n\nt\n\np(An(cid:48)\u2192n, Wn(cid:48)\u2192n(t)| S, \u03b8(cid:96), \u03b8GLM, A\u00acn(cid:48)\u2192n, W \u00acn(cid:48)\u2192n(t)) dWn(cid:48)\u2192n(t) dt\n\n(cid:90) T\n(cid:90) \u221e\n(cid:34) T(cid:89)\nP(cid:88)\n\n\u2212\u221e\n\n0\n\n1\nP\n\nt=1\n\np=1\n\n=\n\n\u2248\n\n(cid:35)\n\n\u02c6\u03c9(p)\nt\n\nPr(An(cid:48)\u2192n),\n\nwhere \u00acn(cid:48) \u2192 n indicates all entries except for n(cid:48) \u2192 n, and the particle weights are obtained by\nrunning a particle \ufb01lter for each assignment of An(cid:48)\u2192n. This allows us to jointly sample An\u2192n(cid:48)\nand Wn\u2192n(cid:48)(t) by \ufb01rst sampling An\u2192n(cid:48) and then Wn\u2192n(cid:48)(t) given An\u2192n(cid:48). By marginalizing out the\nweight trajectory, our algorithm is able to explore the space of adjacency matrices more ef\ufb01ciently.\nWe capitalize on a number of other opportunities for computational ef\ufb01ciency as well. For exam-\nple, if the learning rule factors into independent updates for each Wn(cid:48)\u2192n(t), then we can update\neach synapse\u2019s weight trajectory separately and reduce the particles to one-dimensional objects. In\nour implementation, we also make use of a pMCMC variant with ancestor sampling [18] that sig-\nni\ufb01cantly improves convergence. Any distribution may be used to propagate the particles forward;\nusing the learning rule is simply the easiest to implement and understand. We have omitted a number\nof details in this description; for a thorough overview of particle MCMC, the reader should consult\n[3, 18].\n\n1Note that the particle weights are not the same as the synaptic weights.\n\n5\n\n\fFigure 2: We \ufb01t time-varying weight trajectories to spike trains simulated from a GLM with two neurons\nundergoing no plasticity (top row), an additive, unbounded STDP rule (middle), and a multiplicative, saturating\nSTDP rule (bottom row). We \ufb01t the \ufb01rst 50 seconds with four different models: MAP for an L1-regularized\nGLM, and fully-Bayesian inference for a static, additive STDP, and multiplicative STDP learning rules. In all\ncases, the correct models yield the highest predictive log likelihood on the \ufb01nal 10 seconds of the dataset.\n5 Evaluation\nWe evaluated our technique with two types of synthetic data. First, we generated data from our\nmodel, with known ground-truth. Second, we used the well-known simulator NEURON to simulate\ndriven, interconnected populations of neurons undergoing synaptic plasticity. For comparison, we\nshow how the sparse, time-varying GLM compares to a standard GLM with a group LASSO prior\non the impulse response coef\ufb01cients for which we can perform ef\ufb01cient MAP estimation.\n5.1 GLM-based simulations\nAs a proof of concept, we study a single synapse undergoing a variety of synaptic plasticity rules\nand generating spikes according to a GLM. The neurons also have inhibitory self-connections to\nmimic refractory effects. We tested three synaptic plasticity mechanisms: a static synapse (i.e., no\nplasticity), the unbounded, additive STDP rule given by Equation 3, and the bounded, multiplicative\nSTDP rule given by Equation 4. For each learning rule, we simulated 60 seconds of spiking activity\nat 1kHz temporal resolution, updating the synaptic weights every 1s. The baseline \ufb01ring rates were\nnormally distributed with mean 20Hz and standard deviation of 5Hz. Correlations in the spike timing\nled to changes in the synaptic weight trajectories that we could detect with our inference algorithm.\nFigure 2 shows the true and inferred weight trajectories, the inferred learning rules, and the predictive\nlog likelihood on ten seconds of held out data for each of the three ground truth learning rules. When\nthe underlying weights are static (top row), MAP estimation and static learning rules do an excellent\n\n6\n\n\fFigure 3: Evaluation of synapse detection on a 60 second spike train from a network of 10 neurons undergoing\nsynaptic plasticity with a saturating, additive STDP rule, simulated with NEURON. The sparse, time-varying\nGLM with an additive rule outperforms the fully-Bayesian model with static weights, MAP estimation with L1\nregularization, and simple thresholding of the cross-correlation matrix.\n\njob of detecting the true weight whereas the two time-varying models must compensate by either\nsetting the learning rule as close to zero as possible, as the additive STDP does, or setting the\nthreshold such that the weight trajectory is nearly constant, as the multiplicative model does. Note\nthat the scales of the additive and multiplicative learning rules are not directly comparable since the\nweight updates in the multiplicative case are modulated by how close the weight is to the threshold.\nWhen the underlying weights vary (middle and bottom rows), the static models must compromise\nwith an intermediate weight. Though the STDP models are both able to capture the qualitative\ntrends, the correct model yields a better \ufb01t and better predictive power in both cases.\nIn terms of computational cost, our approach is clearly more expensive than alternative approaches\nbased on MAP estimation or MLE. We developed a parallel implementation of our algorithm to\ncapitalize on conditional independencies across neurons, i.e.\nfor the additive and multiplicative\nSTDP rules we can sample the weights W \u2217\u2192n independently of the weights W \u2217\u2192n(cid:48). On the two\nneuron examples we achieve upward of 2 iterations per second (sampling all variables in the model),\nand we run our model for 1000 iterations. Convergence of the Markov chain is assessed by analyzing\nthe log posterior of the samples, and typically stabilizes after a few hundred iterations. As we scale\nto networks of ten neurons, our running time quickly increases to roughly 20 seconds per iteration,\nwhich is mostly dominated by slice sampling the learning rule parameters. In order to evaluate the\nconditional probability of a learning rule parameter, we need to sample the weight trajectories for\neach synapse. Though these running times are nontrivial, they are not prohibitive for networks that\nare realistically obtainable for optical study of synaptic plasticity.\n5.2 Biophysical simulations\nUsing the biophysical simulator NEURON, we performed two experiments. First, we considered a\nnetwork of 10 sparsely interconnected neurons (28 excitatory synapses) undergoing synaptic plas-\nticity according to an additive STDP rule. Each neuron was driven independently by a hidden\npopulation of 13 excitatory neurons and 5 inhibitory neurons connected to the visible neuron with\nprobability 0.8 and \ufb01xed synaptic weights averaging 3.0 mV. The visible synapses were initialized\nclose to 6.0 mV and allowed to vary between 0.0 and 10.5 mV. The synaptic delay was \ufb01xed at\n1.0 ms for all synapses. This yielded a mean \ufb01ring rate of 10 Hz among visible neurons. Synap-\ntic weights were recorded every 1.0 ms. These parameters were chosen to demonstrate interesting\nvariations in synaptic strength, and as we transition to biological applications it will be necessary to\nevaluate the sensitivity of the model to these parameters and the appropriate regimes for the circuits\nunder study.\nWe began by investigating whether the model is able to accurately identify synapses from spikes, or\nwhether it is confounded by spurious correlations. Figure 3 shows that our approach identi\ufb01es the\n28 excitatory synapses in our network, as measured by ROC curve (Add. STDP AUC=0.99), and\noutperforms static models and cross-correlation. In the sparse, time-varying GLM, the probability\nof an edge is measured by the mean of A under the posterior, whereas in the standard GLM with\nMAP estimation, the likelihood of an edge is measured by area under the impulse response.\n\n7\n\n9mV\fFigure 4: Analogously to Figure 2, a sparse, time-varying GLM can capture the weight trajectories and learning\nrules from spike trains simulated by NEURON. Here an excitatory synapse undergoes additive STDP with a\nhard upper bound on the excitatory postsynaptic current. The weight trajectory inferred by our model with the\nsame parametric form of the learning rule matches almost exactly, whereas the static models must compromise\nin order to capture early and late stages of the data, and the multiplicative weight exhibits qualitatively different\ntrajectories. Nevertheless, in terms of predictive log likelihood, we do not have enough information to correctly\ndetermine the underlying learning rule. Potential solutions are discussed in the main text.\nLooking into the synapses that are detected by the time-varying model and missed by the static\nmodel, we \ufb01nd an interesting pattern. The improved performance comes from synapses that decay\nin strength over the recording period. Three examples of these synaptic weight trajectories are shown\nin the right panel of Figure 3. The time-varying model assigns over 90% probability to each of the\nthree synapses, whereas the static model infers less than a 40% probability for each synapse.\nFinally, we investigated our model\u2019s ability to distinguish various learning rules by looking at a\nsingle synapse, analogous to the experiment performed on data from the GLM. Figure 4 shows\nthe results of a weight trajectory for a synapse under additive STDP with a strict threshold on the\nexcitatory postsynaptic current. The time-varying GLM with an additive model captures the same\ntrajectory, as shown in the left panel. The GLM weights have been linearly rescaled to align with the\ntrue weights, which are measured in millivolts. Furthermore, the inferred additive STDP learning\nrule, in particular the time constants and relative amplitudes, perfectly match the true learning rule.\nThese results demonstrate that a sparse, time-varying GLM is capable of discovering synaptic weight\ntrajectories, but in terms of predictive likelihood, we still have insuf\ufb01cient evidence to distinguish\nadditive and multiplicative STDP rules. By the end of the training period, the weights have saturated\nat a level that almost surely induces postsynaptic spikes. At this point, we cannot distinguish two\nlearning rules which have both reached saturation. This motivates further studies that leverage this\nprobabilistic model in an optimal experimental design framework, similar to recent work by Shababo\net al. [19], in order to conclusively test hypotheses about synaptic plasticity.\n6 Discussion\nMotivated by the advent of optical tools for interrogating networks of synaptically connected neu-\nrons, which make it possible to study synaptic plasticity in novel ways, we have extended the GLM\nto model a sparse, time-varying synaptic network, and introduced a fully-Bayesian inference al-\ngorithm built upon particle MCMC. Our initial results suggest that it is possible to infer weight\ntrajectories for a variety of biologically plausible learning rules.\nA number of interesting questions remain as we look to apply these methods to biological record-\nings. We have assumed access to precise spike times, though extracting spike times from optical\nrecordings poses inferential challenges of its own. Solutions like those of Vogelstein et al. [20]\ncould be incorporated into our probabilistic model. Computationally, particle MCMC could be re-\nplaced with stochastic EM to achieve improved performance [18], and optimal experimental design\ncould aid in the exploration of stimuli to distinguish between learning rules. Beyond these direct ex-\ntensions, this work opens up potential to infer latent state spaces with potentially nonlinear dynamics\nand non-Gaussian noise, and to infer learning rules at the synaptic or even the network level.\nAcknowledgments This work was partially funded by DARPA YFA N66001-12-1-4219 and NSF IIS-\n1421780. S.W.L. was supported by an NDSEG fellowship and by the NSF Center for Brains, Minds, and\nMachines.\n\n8\n\nmV12\fReferences\n[1] Adam M Packer, Darcy S Peterka, Jan J Hirtz, Rohit Prakash, Karl Deisseroth, and Rafael Yuste. Two-\n\nphoton optogenetics of dendritic spines and neural circuits. Nature methods, 9(12):1202\u20131205, 2012.\n\n[2] Daniel R Hochbaum, Yongxin Zhao, Samouil L Farhi, Nathan Klapoetke, Christopher A Werley, Vikrant\nKapoor, Peng Zou, Joel M Kralj, Dougal Maclaurin, Niklas Smedemark-Margulies, et al. All-optical\nelectrophysiology in mammalian neurons using engineered microbial rhodopsins. Nature methods, 2014.\n\n[3] Christophe Andrieu, Arnaud Doucet, and Roman Holenstein. Particle Markov chain Monte Carlo meth-\n\nods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3):269\u2013342, 2010.\n\n[4] Liam Paninski. Maximum likelihood estimation of cascade point-process neural encoding models. Net-\n\nwork: Computation in Neural Systems, 15(4):243\u2013262, January 2004.\n\n[5] Wilson Truccolo, Uri T. Eden, Matthew R. Fellows, John P. Donoghue, and Emery N. Brown. A point\nprocess framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic\ncovariate effects. Journal of Neurophysiology, 93(2):1074\u20131089, 2005.\n\n[6] Ian Stevenson and Konrad Koerding. Inferring spike-timing-dependent plasticity from spike train data. In\n\nAdvances in Neural Information Processing Systems, pages 2582\u20132590, 2011.\n\n[7] Seif Eldawlatly, Yang Zhou, Rong Jin, and Karim G Oweiss. On the use of dynamic Bayesian networks\nin reconstructing functional neuronal networks from spike train ensembles. Neural Computation, 22(1):\n158\u2013189, 2010.\n\n[8] Biljana Petreska, Byron Yu, John P Cunningham, Gopal Santhanam, Stephen I Ryu, Krishna V Shenoy,\nand Maneesh Sahani. Dynamical segmentation of single trials from population neural data. In Neural\nInformation Processing Systems, pages 756\u2013764, 2011.\n\n[9] Mike West, P Jeff Harrison, and Helio S Migon. Dynamic generalized linear models and Bayesian fore-\n\ncasting. Journal of the American Statistical Association, 80(389):73\u201383, 1985.\n\n[10] T. J. Mitchell and J. J. Beauchamp. Bayesian Variable Selection in Linear Regression. Journal of the\n\nAmerican Statistical Association, 83(404):1023\u2014-1032, 1988.\n\n[11] James Robert Lloyd, Peter Orbanz, Zoubin Ghahramani, and Daniel M Roy. Random function priors\nfor exchangeable arrays with applications to graphs and relational data. Advances in Neural Information\nProcessing Systems, 2012.\n\n[12] Natalia Caporale and Yang Dan. Spike timing-dependent plasticity: a Hebbian learning rule. Annual\n\nReview of Neuroscience, 31:25\u201346, 2008.\n\n[13] Erkki Oja. Simpli\ufb01ed neuron model as a principal component analyzer. Journal of Mathematical Biology,\n\n15(3):267\u2013273, 1982.\n\n[14] Daniel E Feldman. The spike-timing dependence of plasticity. Neuron, 75(4):556\u201371, August 2012.\n\n[15] S Song, K D Miller, and L F Abbott. Competitive Hebbian learning through spike-timing-dependent\n\nsynaptic plasticitye. Nature Neuroscience, 3(9):919\u201326, September 2000. ISSN 1097-6256.\n\n[16] Abigail Morrison, Markus Diesmann, and Wulfram Gerstner. Phenomenological models of synaptic\n\nplasticity based on spike timing. Biological cybernetics, 98(6):459\u2013478, 2008.\n\n[17] Anne C Smith and Emery N Brown. Estimating a state-space model from point process observations.\n\nNeural Computation, 15(5):965\u201391, May 2003.\n\n[18] Fredrik Lindsten, Michael I Jordan, and Thomas B Sch\u00a8on. Ancestor sampling for particle Gibbs.\n\nAdvances in Neural Information Processing Systems, pages 2600\u20132608, 2012.\n\nIn\n\n[19] Ben Shababo, Brooks Paige, Ari Pakman, and Liam Paninski. Bayesian inference and online experimental\ndesign for mapping neural microcircuits. In Advances in Neural Information Processing Systems, pages\n1304\u20131312, 2013.\n\n[20] Joshua T Vogelstein, Brendon O Watson, Adam M Packer, Rafael Yuste, Bruno Jedynak, and Liam Panin-\nski. Spike inference from calcium imaging using sequential Monte Carlo methods. Biophysical journal,\n97(2):636\u2013655, 2009.\n\n9\n\n\f", "award": [], "sourceid": 1230, "authors": [{"given_name": "Scott", "family_name": "Linderman", "institution": "Harvard Unviersity"}, {"given_name": "Christopher", "family_name": "Stock", "institution": "Harvard College"}, {"given_name": "Ryan", "family_name": "Adams", "institution": "Harvard University"}]}