{"title": "Biologically Plausible Local Learning Rules for the Adaptation of the Vestibulo-Ocular Reflex", "book": "Advances in Neural Information Processing Systems", "page_first": 961, "page_last": 968, "abstract": null, "full_text": "Biologically Plausible Local Learning Rules for \nthe Adaptation of the Vestibulo-Ocular Reflex \n\nOlivier Coenen* \n\nTerrence J. Sejnowski \n\nComputational Neurobiology Laboratory \n\nHoward Hughes Medical Institute \n\nThe Salk Institute \nP.O.Box 85800 \n\nSan Diego, CA 92186-5800 \n\nStephen G. Lisberger \n\nDepartment of Physiology \n\nW.M. Keck Foundation Center \nfor Integrative Neuroscience \n\nUniversity of California, \nSan Fransisco, CA, 94143 \n\nAbstract \n\nThe vestibulo-ocular reflex (VOR) is a compensatory eye movement that \nstabilizes images on the retina during head turns. Its magnitude, or gain, \ncan be modified by visual experience during head movements. Possible \nlearning mechanisms for this adaptation have been explored in a model \nof the oculomotor system based on anatomical and physiological con(cid:173)\nstraints. The local correlational learning rules in our model reproduce the \nadaptation and behavior of the VOR under certain parameter conditions. \nFrom these conditions, predictions for the time course of adaptation at \nthe learning sites are made. \n\n1 \n\nINTRODUCTION \n\nThe primate oculomotor system is capable of maintaining the image of an object on the \nfovea even when the head and object are moving simultaneously. The vestibular organs \nprovide information about the head velocity with a short delay of 14 ms but visual Signals \nfrom the moving object are relatively slow and can take 100 ms to affect eye movemen.ts. \nThe gain, a, of the VOR, defined as minus the eye velocity over the head velocity (-if h), \ncan be modified by wearing magnifying or diminishing glasses (figure 1). VOR adaptation, \nabsent in the dark, is driven by the combination of image slip on the retina and head turns. \n\n\u00b7University of California, San Diego. Dept. of Physics. La Jolla, CA, 92037. Email address: \n\noli vier@helmholtz.sdsc.edu \n\n961 \n\n\f962 \n\nCoenen, Sejnowski, and Lisberger \n\nDuring head turns on the first day of wearing magnifying glasses, the magnified image of \nan object slips on the retina. After a few days of adaptation, the eye velocity and hence the \ngain of the VOR increases to compensate for the image magnification. \n\nWe have constructed a model of the VOR and smooth pursuit systems that uses biologically \nplausible local learning rules that are consistent with anatomical path ways and physiological \nrecordings. The learning rules in the model are local in the sense that the adaptation of a \nsynapse depends solely on signals that are locally available. A similar model with different \nlocal learning rules has been recently proposed (Quinn et at., Neuroscience 1992). \n\nxl.O \n\nSpectacles \n\noff \n~ \n\no . \n\nGain = 1.01 + 0 .68(1 . e-<1020 t) \n\nGain = 1.01 + 0.68 Ie -0._ tl \n\nxl.O \n\nSpectacles \n\non \n9 \n, , \n, \n\nl\u00b7 \n\n1 B \n\n1 I; \n\n1.4 \n\n1.2 \n\n<.:) \n\n\u2022 \n\nz < 1.0 \n~ :: r ~--:---:---:----:-----m. __ ___ \n0.6 [ + \n\nGain\" = O.~ + 027 Ie\" -0 1J t) \n\nII \n\n\" \n\nxO.5 \n\nSpectaCles \n\n, \n\n~--------\n\n\" \n--\n\n\" \n--------\n\u2022 \n\n\u2022 \n\nf \n\nxO.5 \n\nSpectacles \n\non \noff \n, \n,L-....L.---.J'_....l,_....L, _...L' _..L.' _.I..-' \no 23456780 234567 \n\n\u2022 \n\nI \n\n, \n\nI \n\nI \n\n! \n\n! \n\nTIME IDaysl \n\nFigure 1: Tune course of the adapting VOR and its recovery of gain in monkeys exposed to the long(cid:173)\nterm influence of magnifying (upper curves) and diminishing (lower curves) spectacles. Different \nsymbols obtained from different animals, demonstrating the consistency of the adaptive change. From \nMelvill Jones (1991), selected from Miles and Eighmy (1980). \n\n2 THEMODEL \n\nFeedforward and recurrent models of the VOR have been proposed (Fujita, 1982; Galiana, \n1986; Kawato and Gomi, 1992; Quinn et al., 1992; Arnold and Robinson, 1992; Lisberger \nand Sejnowski, 1992). In this paper we study a static and linear version of a previously \nstudied recurrent network model of the VOR and smooth pursuit system (Lisberger, 1992; \nLisberger and Sejnowski, 1992; Viola, Lisberger and Sejnowski, 1992). The time delays \nand time constants associated with nodes in the network were eliminated so that the time \ncourse of the VOR plasticity could be more easily analyzed (figure 2). \n\nThe model describes the system ipsilateral to one eye. The visual error, which carries the \nimage retinal slip velocity signal, is a measure of the performance of both the VOR and \nsmooth pursuit system as well as the main error signal for learning. The value at each node \nrepresents changes in its firing rate from its resting firing rate. The transformation from the \nrate of firing of premotor signal (N) to eye velocity is represented in the model by a gain \n\n\fBiologically Plausible Local Learning Rules for Adaptation of Vestibulo-Ocular Reflex \n\n963 \n\no : Gains \n\nP : Purkinje Cell \nN : Vestibular Nucleus \ng : Desired gain \n\nVisual error: mossy fibers \n\n-(g h + e) \n\n\u2022 h \n\nInhibitory \n\n\u2022 e \n\neye velocity \n\nVisual error: climbing fibers \n\nFigure 2: Diagram of the VOR and smooth pursuit model. The input and output of the model are, \nrespectively, head velocity and eye velocity. The model has three main parts: the node P represents \nan ensemble of Purkinje cells from the ventral paraflocculus of the cerebellum, the node N represents \nan ensemble of flocculus-target neurons in the vestibular nucleus, and the visual inputs which provide \nthe visual error signals in the mossy and climbing fibers. The capital letter gains A and D, multiplying \nthe input signals to the nodes, are modified according to their learning rules. The lower case letters \nb, v, and 9 are also multiplicative gains, but remain constant during adaptation. The traces represent \nhead and eye velocity modulation in time. The visual error signal in the climbing fibers drives learning \nin node N but does not constitute one of its inputs in the present model. \n\nof -1. The gain of the VOR in this model is given by ~=:. We have not modeled the \nneural integrator that converts eye velocity commands to eye position signals that drive the \nmotoneurons. \n\n3 LEARNING RULES \n\nWe have adopted the learning rules proposed by Marr (1969), Albus (1971) and Ito (1970) \nfor adaptation in the cerebellum and by Lisberger (1988), Miles and Lisberger (1981) for \nplasticity in the brain stem (figure 3). These are variations of the delta rule and depend on \nan explicit representation of the error signal at the synapses. \nLong term depression at mossy fiber synapses on Purkinje cells has been observed in \nvitro under simultaneous stimulation of climbing fibers and mossy fibers (Ito, Sakurai and \nTongroach, 1982). In addition, we have included a learning mechanism for potentiation \nof mossy fiber head velocity inputs under concurrent mossy fiber visual and head velocity \ninputs. Although the climbing fiber inputs to the cerebellum were not directly represented \nin this model (figure 2), the image velocity signal carried by the mossy fibers to P was used \nin the model to achieve the same result. \nThere is good indirect evidence that learning also occurs in the vestibular nucleus. We \nhave adopted the suggestion of Lis berger (1988) that the effectiveness of the head velocity \ninput to some neurons in the vestibular nucleus may be modified by head velocity input in \n\n\f964 \n\nCoenen, Sejnowski, and Lisberger \n\n~ \n\n-\n\nLearning \n\nRate \n\n( \n\nx \n\nInPut) \nSignal \n\nx \n\n(Error) \n\nSignal \n\nCerebellum (P): \n\nA \n\nVestibular nucleus (N): \n\nHead ) ( Mossy fiber \n\nVisual signal \n\n) \n\nVelocity \n\nx \nqA X ( \nqA x h x -v(gh + e) \nqA X h x -v[(g - D)h + P] \n\n-\n-\nex: h2 \n\nb \n\nqD X ( \n\nHead \n\nVelocity \n\n) \n\n(Climbing fiber \nVisual signal \n\nx \n\nqD x h x [(1 - q)(gh + e) - qP] \nqD X h x [(1 - q)(g - D)h + (1 - 2q)P] \n\n-\n-\noc h2 \n\nwhere \n\nP \n\nA - bD - (g - D)v . \nh \n\nI-b+v \n\nPurkinje ) \n\nSignal \n\nFigure 3: Learning rules for the cerebellum and vestibular nucleus. The gains A and D change \naccording to the correlation of their input signal and the error signal to the node, as shown for ~ at \nthe top. The parameter q determines the proportion of learning from Purkinje cell inputs compared \nto learning from climbing fiber inputs. When q = I, only Purkinje cell inputs drive the adaptation at \nnode N; if q = 0, learning occurs solely from climbing fiber inputs. \n\nassociation with Purkinje cells firing. We have also added adaptation from pairing the head \nvelocity input with climbing fiber firing. The relative effectiveness of these two learning \nmechanisms is controlled by the parameter q (figure 3). \n\nLearning for gain D depends on the interplay between several signals. If the VOR gain is \ntoo small. a rightward head turn P (positive value for head velocity) results in too small a \nleftward eye turn (a negative value for eye velocity). Consequently, the visual scene appears \nto move to the left (negative image slip). P then fires below its resting level (negative) and \nits inhibitory influence on N decreases so that N increases its firing rate (figure 4 bottom \nleft). This corrects the VOR gain and increases gain D according to figure 3. Concurrently, \nthe climbing fiber visual signal is above resting firing rate (positive) which also leads to an \nincrease in gain D. \n\nSince the signal passing through gain A has an inhibitory influence via Ponto N, decreasing \ngain A has the opposite effect on the eye velocity as decreasing gain D. Hence, if the VOR \nis too small we expect gain A to decrease. This is what happens during the early phase of \nlearning (figure 4 top left). \n\n4 RESULTS \n\nFinite difference equations of the learning rules were used to calculate changes in gains A \nand D at the end of each cycle during our simulations. A cycle was defined as one biphasic \n\n\fBiologically Plausible Local Learning Rules for Adaptation of Vestibulo-Ocular Reflex \n\n965 \n\nDesired gain 9 = 1.6 \n\nMagnitude \n\nG \n\nD \n\nA \n\n20 \n\n40 \n\n60 \n\n80 \n\n10lime \n\nMagnitude \n\n2 \nt.75 \n1.5 \n1.25 \n1 \n0.75 \n0.5 \n0.25 \n0 \n\nG \n\nD \n\nA \n\n2000 \n\n4000 \n\n6000 \n\n8000 \n\n10(J;ime \n\n1.5 / \n\n2 \n1.75 \n\nt.25 \n1 \n0.7!1 \n0.5 \n0.25 \n0 \n\nAmplitude \n\n2 \n\n1.5 / \n\n1 \n\n0.5 \n\nA, D & VOR gain G vs time \n\nAmplitude \n\n2 \n\n1.5 \n\n1 \n\n0.5 \n\n~.5 \n\n\u00b71 \n\nN \n\nP \n\n100 Time \n\n2000 \n\n4000 \n\n6000 \n\nN \n\nP \n\n~ 40 \n\n~.5 \n\n60 \n\n80 \n\n\u00b71 \n\nP & N responses to a head turn during learning vs time \n\nFigure 4: Simulation of change in gain from 1.0 to 1.6. Top: Short-term (left) and long-term (right) \nadaptation of the gains A, D and G. Bottom: Changes on two time scales of P and N responses to a \nhead turn of amplitude 1 during learning. The parameters were v = 1.0, b = .88, T = .!lA = 10. , \nand q = .01. \n\nf1D \n\nhead velocity input as shown in figure 2. We assumed that the learning rates were so small \nthat the changes in gains, and hence in the node responses, were negligibly small during \neach iteration. This allowed the replacement ~f A(t) .and D(t) by their values obtained on \nthe previous iteration for the calculations of A and D. The period of the iteration as well \nas the amplitude of the head velocity input were chosen so that the integral of the head \nvelocity squared over one iteration equaled l. \n\nFor the simulations shown in figure 4 the gain G of the VOR increased monotonically from \n1 to reach the desired value 1.6 within 60 time steps. This rapid adaptation was mainly \ndue to a rapid decrease in A, as expected from the local learning rule (figure 3), since the \nlearning rate 'f/A was greater than the learning rate 'f/D. Over a longer time period, learning \nwas transferred from A to D: D increased from 1 to reach its final value 1.6 while the VOR \ngain stayed constant. Transfer of learning occurs when P fires in conjunction with a head \nturn. P can have an elevated firing rate even though the visual error signal is zero (that is, \neven if the VOR gain G has reached the desired gain g) because of the difference between \nits two other inputs: the head velocity input through A and the eye velocity feedback input \nthrough b. It is only when these two inputs become equal in.amplit~lde that P firing goes \nto zero. It can be shown that when learning settles (when D and A equal zero) D = g, \nA = bg, and P = O. With these values for A and D, the two other inputs to P are indeed \nequal in amplitude: one equals Ah, while the other equals b( -1 )Dh. During the later \npart of learning, gain A is driven in the opposite direction (increase) than during the earlier \n\n\f966 \n\nCoenen, Sejnowski, and Lisberger \n\npart ( decrease). This comes from a sign reversal of the visual error input to P. After the \nfirst 60 time steps, the gain has reached the desired gain due to a rapid decrease in A, this \nmeans that any subsequent increase in D, due to transfer of learning as explained above, \nwill cause the gain of the VOR G to become larger than the desired gain g, hence the visual \nerror changes sign. In order to compensate for this small error, gain A increases promptly, \nkeeping G very close to the desired gain. This process goes on until A and D reach their \nequilibrium values stated above. \n\nThe short and long-term changes in P and N responses to a velocity step are also shown. \nAs the firing of P decreased with the adaptation of A, the firing rate of N increased to the \nright level. \n\n5 OVERSHOOT OF THE VOR GAIN G \n\nIn this section we show that for some ranges of the learning parameters, the gain G in \nthe model overshoots the desired value g. Since an overshoot is not observed in animals \n(figure I), this provides constraints on the parameters. The parameter q in the learning rule \nfor the vestibular nucleus (node N, gain D), determines the proportion of learning from \nPurkinje cell inputs compared to learning from climbing fiber inputs. When q = 1, only \nPurkinje cell inputs drive the adaptation at node N; if q = 0, learning at N occurs solely \nfrom climbing fiber inputs. These two inputs have quite different effects on learning as \ndiffer from \u00b0 if q = 0. The gain has an overshoot for any value of q different than 0, as \nshown in figure 5. Asymptotically, P goes to 0, and D goes to 9 if q = 1; and P can only \nand a larger r. One possibility is that q is chosen close to \u00b0 and r > I, that is TJA > 7JD. \n\nshown in figure 6. Nevertheless, its amplitude is only significant for a limited extent in the \nparameter space of q and r (graph of figure 6). The overshoot is reduced with a smaller q \n\nThese conditions were used to choose parameter values in the simulations (figure 4). \n\n6 DISCUSSION AND CONCLUSION \n\nThe VOR model analyzed here is a static model without time delays and multiple time \nscales. We are currently studying how these factors affect the time course of learning in a \ndynamical model of the VOR and smooth pursuit. \nIn our model, learning occurs in the dark if P #- 0, which has not been observed in animals. \nOne way to avoid learning in the dark when P is firing would be to gate the learning by a \nvisual input, such as that provided by climbing fibers. \n\nThe responses of vestibular afferents to head motion can be classified into two categories: \nphase-tonic and tonic. In this model, only the tonic afferents were represented. Both \nafferent types encode head velocity, while the phasic-tonic responds to head acceleration as \nwell. The steady state VOR gain can also be changed by altering the relative proportions \nof phasic and tonic afferents to the Purkinje cells (Lisberger and Sejnowski, 1992). We are \ncurrently investigating learning rules for which this occurs. \n\nThe model predicts that adaptation in the cerebellum is faster than in the vestibular nucleus, \nand that learning in the vestibular nucleus is mostly driven by the climbing fiber error \nsignals. \n\nThe model shows how the dynamics of the whole system can lead to long-term adaptation \n\n\fBiologically Plausible Local Learning Rules for Adaptation of Vestibulo-Ocular Reflex \n\n967 \n\nDesired gain g = 1. 6 \n\nq = 1 \n\nMagnibtde \n\n2.5 \n\nG \n\n100 \n\n200 \n\n300 \n\n0.5 \n\n0 \n\nAmplitude \n\n1.5 \n\n0 \n\nA \n\n1 \n1.75 \n1.5 \n1.15 \n1 \n0.75 \n0.5 \n0.15 \n0 \n\nG \n\n0 \n\nA \n\n200 \n\n300 \n\n400 \n\n50lime \n\nq=O \n\nMagnibtde \n\nsooTime \n\n400 \nA, D & VOR gain G vs time \n\n100 \n\nAmplitude \n\n2.5 \n\n1 \n\nN \n\nN \n\n300 \n\n400 \n\n500 Time \n\n100 \n\nP \n\n200 \n\n300 \n\n400 \n\n500 Time \n\n\u00b71 \n\n\u00b71 \n\nP & N responses to a head turn during learning vs time \n\nFigure 5: Effect of q on learning curves for gain increase. Left: q = 1 leads to an (wershoot in \nthe VOR gain G above the desired gain. D increases up to the desired gain, P starts from 0 and \nasymptotically goes back to 0; both indicate that learning is totally transferred from P to N. Right: \nWith q = 0, there is no overshoot in the VOR gain, but since A decreases to a constant value and \nD only increases very slightly, learning is not transfered. Consequently, P firing rate stays constant \nafter an initial drop. \n\nE \n\n(I-b+v) (D \n(I-b) \n\n) \n\nq \n\n- 9 (2q-I)-rv \n\nFigure 6: Overshoot f. of the VOR gain G as a function of q and r. The parameter q is the proportion \nof learning to node N (vestibular nucleus), coming from the P node (cerebellum) compared to learning \nfrom climbing fibers. The parameter T is the ratio of the learning rates TJA and TJD. No overshoot is \nseen in animals, which restricts the parameters space of q and r for the model to be valid. Note that \nthe overshoot diverges for some parameter values.' \n\n10 \n\nwhich differs from what may be expected from the local learning rules at the synapses \nbecause of differences in time scales and shifts of activity in the system during learning. \nThis may reconcile apparently contradictory evidence between local learning rules ob(cid:173)\nserved in vitro (Ito, 1970) and the long-term adaptation seen in vivo in animals (Miles and \nLisberger, 1981). \n\n\f968 \n\nCoenen, Sejnowski, and Lisberger \n\nAcknowledgments \n\nO.c. was supported by NSERC during this research. \n\nReferences \n\nAlbus, J. S. (1971). A theory of cerebellar function. Math. Biosci., 10:25-61. \n\nArnold, D. B. and Robinson, D. A. (1992). A neural network model of the vestibulo-ocular reflex using a local \n\nsynaptic learning rule. Phil. Trans. R. Soc. Lond. B, 337:327-330. \n\nFujita, M. (1982). Simulations of adaptive modification of the vestibulo-ocular reflex with an adaptive filter model \n\nof the cerebellum. Biological Cybernetics, 45:207-214. \n\nGaliana, H. L. (1986). A new approach to understanding adaptive visual-vestibular interactions in the central \n\nnervous system. Journal of Neurophysiology, 55:349-374. \n\nIto, M. (1970). Neurophysiological aspects of the cerebellar motor control system. Int.J.Neurol., 7:162-176. \n\nIto, M., Sakurai, M., and Tongroach, P. (1982). Climbing fibre induced depression of both mossy fibre respon(cid:173)\n\nsiveness and glutamate sensitivity of cerebellar purkinje cells. J. Physiol. Lond., 324:113-134. \n\nKawato, M. and Gomi, H. (1992). The cerebellum and VORlOKR learning models. Trends in Neuroscience, \n\n15 :445-453. \n\nLisberger, S. G. (1988). The neural basis for learning of simple motor skills. Science, 242:728-735. \n\nLisberger, S. G. (1992). Neural basis for motor learning in the vestibulo-ocularreflex ofprimates:IV. The sites of \n\nlearning. In preparation. \n\nLisberger, S. G. and Sejnowski, T. J. (1992). Computational analysis suggests a new hypothesis for motor learning \n\nin the vestibulo-ocular reflex. Technical Report 9201, INC, Univ. of California, San Diego. \n\nMarr, D. (1969). A theory of cerebellar cortex. J. Physiol., 202:437-470. \n\nMelviIl Jones, G. M. (1991). The Vestibular Contribution, volume 8 of Vision and Visual Dysfunction, chapter 2, \n\npages 293-303. CRC Press, Inc., Boston. General Editor: J. R. Cronly-Dillon. \n\nMiles, E A. and Eighmy, B. B. (1980). Long-term adaptive changes in primate vestibulo-ocular reflex.l. Be(cid:173)\n\nhavioural observations. Journal of Neurophysiology, 43:140&-1425. \n\nMiles, F. A. and Lisberger, S. G. (1981). Plasticity in the vestibulo-ocular reflex: A new hypothesis. Ann. Rev. \n\nNeurosci., 4:273-299. \n\nQuinn, K. J., Baker, J., and Peterson, B. (1992). Simulation of cerebellar-vestibular interactions during VOR \n\nadaptation. In Program 22nd Annual Meeting. Society for Neuroscience. \n\nQuinn, K. J., Schmajuk, N., Jain, A., Baker, J. E, and Peterson, B. W. (1992). Vestibuloocular reflex arc analysis \n\nusing an experimentally constrained network. Biologtcal Cybernetics, 67: 113-122. \n\nViola, P. A., Lisberger, S. G., and Sejnowski, T. J. (1992). Recurrent eye tracking network using a distributed \nrepresentation of image motion. In Moody, 1. E., Hansen, S. J., and Lippman, R. P., editors, Advances in \nNeural Information Processing Systems 4, San Mateo. IEEE, Morgan Kaufmann Publishers. \n\n\f", "award": [], "sourceid": 704, "authors": [{"given_name": "Olivier", "family_name": "Coenen", "institution": null}, {"given_name": "Terrence", "family_name": "Sejnowski", "institution": null}, {"given_name": "Stephen", "family_name": "Lisberger", "institution": null}]}