{"title": "Ocular Dominance and Patterned Lateral Connections in a Self-Organizing Model of the Primary Visual Cortex", "book": "Advances in Neural Information Processing Systems", "page_first": 109, "page_last": 116, "abstract": null, "full_text": "Ocular Dominance and Patterned Lateral \n\nConnections in a Self-Organizing Model of the \n\nPrimary Visual Cortex \n\nJoseph Sirosh and Risto Miikkulainen \n\nDepartment of Computer Sciences \n\nUniversity of Texas at Austin, Austin, 'IX 78712 \n\nemail:  sirosh.risto~cs.utexas.edu \n\nAbstract \n\nA neural network model for the self-organization of ocular dominance and \nlateral connections from binocular input is presented.  The self-organizing \nprocess results in a network where (1) afferent weights of each neuron or(cid:173)\nganize into smooth hill-shaped receptive fields primarily on one of the reti(cid:173)\nnas, (2) neurons with common eye preference form connected, intertwined \npatches, and (3) lateral connections primarily link regions of the same eye \npreference.  Similar self-organization of cortical  structures has  been  ob(cid:173)\nserved  experimentally in strabismic kittens.  The model  shows how  pat(cid:173)\nterned lateral connections in  the cortex may  develop based on correlated \nactivity and explains why lateral connection patterns follow receptive field \nproperties such as ocular dominance. \n\nIntroduction \n\n1 \nLateral  connections  in  the  primary  visual  cortex  have  a patterned  structure  that  closely \nmatches the response properties of cortical cells (Gilbert and Wiesel 1989; Malach et al.1993). \nFor example, in the normal visual cortex, long-range lateral connections link areas with sim(cid:173)\nilar orientation preference (Gilbert and Wiesel  1989).  Like cortical response properties, the \nconnectivity pattern is highly plastic in early development and can be altered by experience \n(Katz and Callaway 1992). In a cat that is brought up squint-eyed from birth, the lateral con(cid:173)\nnections link areas with the same ocular dominance instead of orientation (Lowel and Singer \n1992). Such patterned lateral connections develop at the same time as the orientation selectiv(cid:173)\nity and ocular dominance itself (Burkhalter et al.1993; Katz and Callaway 1992).  Together, \n\n\f110 \n\nJoseph  Sirosh,  Risto  Miikkulainen \n\nthese observations suggest that the same experience-dependent process drives the develop(cid:173)\nment of both cortical response properties and lateral connectivity. \n\nSeveral computational models have  been  built to  demonstrate  how orientation preference, \nocular dominance,  and retinotopy can  emerge  from  simple  self-organizing processes  (e.g. \nGoodhill1993; Miller 1994; Obermayer et al.1992; von der Malsburg 1973). These models \nassume  that the neuronal response properties are primarily determined by the afferent con(cid:173)\nnections,  and concentrate only on the self-organization of the afferent  synapses  to  the cor(cid:173)\ntex.  Lateral interactions between neurons are abstracted into simple mathematical functions \n(e.g. Gaussians) and assumed to be uniform throughout the network; lateral connectivity is not \nexplicitly taken into account.  Such models do not explicitly replicate the activity dynamics \nof the visual cortex, and therefore can make only limited predictions about cortical function. \n\nWe  have previously shown  how  Kohonen's self-organizing feature  maps  (Kohonen  1982) \ncan  be generalized to include self-organizing lateral connections and recurrent activity dy(cid:173)\nnamics (the Laterally Interconnected Synergetically Self-Organizing Map (LISSOM); Sirosh \nand Miikkulainen 1993, 1994a), and how the algorithm can model the development of ocu(cid:173)\nlar dominance columns and patterned lateral connectivity with abstractions of visual input. \nLISSOM is a low-dimensional abstraction of cortical self-organizing processes and models a \nsmall region of the cortex where all neurons receive the same input vector.  This paper shows \nhow realistic, high-dimensional receptive fields develop as part of the self-organization, and \nscales up the LISSOM approach to large areas of the cortex where different parts of the corti(cid:173)\ncal network receive inputs from different parts of the receptor surface.  The new model shows \nhow  (1) afferent receptive fields  and ocular dominance columns develop  from  simple reti(cid:173)\nnal images,  (2) input correlations affect the wavelength of the ocular dominance columns and \n(3) lateral connections self-organize cooperatively and simultaneously with ocular dominance \nproperties.  The model suggests new computational roles for lateral connections in the cortex, \nand suggests that the visual cortex maybe maintained in a continuously adapting equilibrium \nwith the visual input by co adapting lateral and afferent connections. \n\n2  The LISSOM Model of Receptive Fields and Ocular Dominance \n\nThe LISSOM network is a sheet of interconnected neurons (figure 1).  Through afferent con(cid:173)\nnections, each neuron receives input from two \"retinas\".  In addition, each neuron has recip(cid:173)\nrocal excitatory and inhibitory lateral connections with other neurons. Lateral excitatory con(cid:173)\nnections are short-range, connecting only close neighbors. Lateral inhibitory connections run \nfor long distances, and may even implement full connectivity between neurons in the network. \n\nNeurons  receive afferent  connections  from  broad overlapping patches  on  the retina called \nanatomical  receptive fields,  or RFs.  The  N  x  N  network is projected on to  each  retina of \nR  x R receptors, and each neuron is connected to receptors in a square area of side s around \nthe projections.  Thus, neurons receive afferents  from corresponding regions of each retina. \nDepending on the location of the projection, the number of afferents to a neuron from each \nretina could vary from ts  x  ~s (at the comers) to s  x s (at the center). \nThe external and lateral weights are organized through an unsupervised learning process.  At \neach training step, neurons start out with zero activity. The initial response TJij  of neuron (i, j) \n\n\fOcular Dominance and Patterned Lateral Connections \n\n111 \n\nLoft  _ . .  \n\nfllgIIl Roll .. \n\nFigure 1:  The Receptive-Field LISSOM architecture. The afferent and lateral connectionsof a single \nneuron in the liSSOM network are shown. All connection weights are positive. \n\nis based on the scalar product \n\nTJij  =  (T (L eabJJij ,ab + L eCdJJij,Cd)  , \n\na,b \n\nc,d \n\n(1) \n\nwhere eab and ecd are the activations of retinal receptors (a, b)  and (c, d) within the receptive \nfields  of the neuron in each retina,  JJij,ab  and JJij,cd  are the corresponding afferent weights, \nand  (T  is a piecewise linear approximation of the familiar sigmoid activation function.  The \nresponse evolves over time through lateral interaction.  At each time step,  the neuron com(cid:173)\nbines the above afferent activation I:: eJJ  with lateral excitation and inhibition: \n\nTJij(t)  =  (T  (L eJJ + \"Ie L Eij,kITJkl(t  - 1) - \"Ii L Iij,klTJkl(t  - 1)) , \n\n(2) \n\nk,1 \n\nk,1 \n\nwhere Eij,kl is the excitatory lateral connection weight on the connection from neuron (k, l) \nto neuron  (i, j),  Iij,kl  is the inhibitory connection weight, and  TJkl (t  - 1)  is the activity of \nneuron  (k, I)  during the previous time step.  The constants \"Ie  and \"Ii  determine the relative \nstrengths of excitatory and inhibitory lateral interactions.  The activity pattern starts out dif(cid:173)\nfuse and spread over a substantial part of the map, and converges iteratively into stable focused \npatches of activity, or activity bubbles.  After the-activity has settled, typically in a few  iter(cid:173)\nations of equation 2, the connection weights of each neuron are modified.  Both afferent and \nlateral weights adapt according to the same mechanism:  the Hebb rule, normalized so that the \nsum of the weights is constant: \n\nWij,mn  t  + vt  -\n\n( \n\nr  )  _ \n\nWij,mn(t) + CtTJijXmn \n\nwmn [Wij ,mn  t  + CtTJijXmn \n'\"\" \n\n(  ) \n\n1 ' \n\n(3) \n\nwhere TJij  stands for the activity of neuron (i, j) in the final activity bubble, Wij,mn is the affer(cid:173)\nent or lateral connection weight (JJ,  E or I), Ct  is the learning rate for each type of connection \n(Ct a for afferent weights, Ct E  for excitatory, and Ct I  for inhibitory) and X mn is the presynaptic \nactivity (e for afferent,  TJ  for lateral). \n\n\f112 \n\nJoseph  Sirosh,  Risto  Miikkulainen \n\n\" \n\n(a) Random Initial Weights \n\n(b) Monocular RF \n\n(c) Binocular RF \n\nFigure 2:  Self-organization of the afferent input weights into receptive fields.  The afferent weights \nof a neuron at position (42,39) in  a 60  x  60 network are shown before (a)  and after self-organization \n(b).  This particular neuron becomes monocular with strong connections to the right eye, and weak con(cid:173)\nnections to the left.  A neuron at position (38, 23)  becomes binocular with appoximately equal weights \nto both eyes (c). \n\nBoth excitatory and  inhibitory lateral connections  follow  the  same  Hebbian  learning  pro(cid:173)\ncess  and strengthen by correlated activity.  The short-range excitation keeps  the activity of \nneighboring neurons  correlated,  and as self-organization progresses,  excitation and inhibi(cid:173)\ntion strengthen in  the vicinity of each neuron.  At longer distances,  very few  neurons have \ncorrelated activity and therefore most long-range connections become weak.  Such weak con(cid:173)\nnections are eliminated, and through weight normalization, inhibition concentrates in a closer \nneighborhood of each neuron.  As a result, activity bubbles become more focused and local, \nweights change in smaller neighborhoods, and receptive fields become better tuned to local \nareas of each retina. \n\nThe input to the model consists of gaussian spots of \"light\" on each retina: \n\n_ \n\nt \n<\"x,y  - exp  -\n\n((x - xd 2 + (y - Yi)2) \n\nu 2 \n\n(4) \n\nwhere ex,y  is the activation of receptor (x, V),  u 2  is a constant determining the width of the \nspot, and (Xi,Yi):  0 ~ xi, Yi  < R its center.  At each input presentation, one spot is randomly \nplaced at (Xi ,Yi) in  the left retina, and a second spot within a radius of p  x  RN of (Xi, yd \nin the right retina.  The parameter p  E  [0, 1]  specifies the spatial correlations between spots \nin the two retinas, and can be adjusted to simulate different degrees of correlations between \nimages in the two eyes. \n\n3  Simulation results \nTo  see how correlation between the input from the two eyes affects the columnar structures \nthat develop, several simulations were run with different values of p.  The afferent weights of \nall neurons were initially random (as shown in figure 2a), with the total strength to both eyes \nbeing equal. \n\nFigures 2b,c show  the final  afferent receptive fields  of two  typical neurons in a simulation \nwith  p  = 1.  In  this case,  the inputs were  uncorrelated,  simulating perfect strabismus.  In \nthe early stages of such simulation, some of the neurons randomly develop a preference for \none eye or the other.  Nearby neurons will tend to share the same preference because lateral \n\n\fOcular Dominance and Patterned Lateral Connections \n\n113 \n\n(a) Connections of a Monocular Neuron \n\n(b) Connections of a Binocular Neuron \n\nFigure 3:  Ocular dominance and lateral connection patterns. The ocular dominance of a neuron is \nmeasured as the difference in total afferent synaptic weight from each eye to the neuron.  Each neuron \nis  labeled with  a grey-scale value (black  ~ white) that represents continuously changing eye prefer(cid:173)\nence from exclusive left through binocular to exclusive right.  Small white dots indicate the lateral input \nconnections to the neuron marked with  a big white dot.  (a) The surviving lateral connections of a left \nmonocular neuron predominantly link areas of the same ocular dominance.  (b) The lateral connections \nof a binocular neuron come from both eye regions. \n\nexcitation keeps neural activity partially correlated over short distances.  As self-organization \nprogresses, such preferences are amplified, and groups of neurons develop strong weights to \none eye.  Figure 2b shows the afferent weights of a typical monocular neuron. \n\nThe extent of activity correlations on the network detennines the size of the monocular neu(cid:173)\nronal groups.  Farther on the map,  where the activations are anticorrelated due to lateral in(cid:173)\nhibition, neurons will develop eye preferences to  the opposite eye.  As  a result,  alternating \nocular dominance patches develop over the map, as shown in figure 3.1  In areas between oc(cid:173)\nular dominance patches, neurons will develop approximately equal strengths to both eyes and \nbecome binocular, like the one shown in figure 2e. \n\nThe width and number of ocular dominance columns in the network (and therefore, the wave(cid:173)\nlength of ocular dominance) depends on the input correlations (figure 4).  When inputs in the \ntwo eyes become more correlated (p  < 1), the activations produced by the two inputs in the \nnetwork overlap closely and activity correlations become shorter range.  By Hebbian adapta(cid:173)\ntion, lateral inhibition concentrates in the neighborhood of each neuron, and the distance at \nwhich activations becomes anticorrelated decreases.  Therefore,  smaller monocular patches \ndevelop, and the ocular dominance wavelength decreases.  Similar dependence was  very re(cid:173)\ncently observed in the cat primary visual cortex (LoweI1994).  The LISSOM model demon(cid:173)\nstrates that the adapting lateral interactions and recurrent activity dynamics regulate the wave(cid:173)\nlength, and suggests how these processes help the cortex develop feature detectors at a scale \n\n1 For a thorough treatment of the mathematical principles underlying the development of ocular dom(cid:173)\n\ninance columns, see (GoodhillI993; Miller et al.1989; von der Malsburg and Singer 1988). \n\n\fJoseph  Sirosh,  Risto  Miikkulainen \n\n114 \n\n- 0 \n-0 \n\n(a) Strabismic case \n\n(b ) Normal case \n\nFigure 4:  Ocular dominance wavelength in strabismic and normal models. In the strabismic case, \nthere are no  between-eye correlations (p  =  1), and broad ocular dominance columns are produced (a). \nWith normal, partial between-eye correlations (p  = 0.45 in this example), narrower stripes are formed \n(b).  As  a result, there are more ocular dominance columns in the normal case and the ocular dominance \nwavelength is smaller. \n\nthat matches the input correlations. \n\nAs eye preferences develop,  left or right eye input tends to cause activity only in the left or \nright ocular dominance patches.  Activity patterns in areas of the network with the same oc(cid:173)\nular dominance tend to be highly correlated because they are caused by the same input spot. \nTherefore,  the long-range lateral connections between similar eye preference areas  become \nstronger,  and those between  opposite areas  weaker.  After the weak  lateral connections are \neliminated,  the initially wide-ranging connections are pruned,  and eventually only connect \nareas  of similar ocular dominance as shown in figure  3.  Binocular neurons between ocular \ndominance patches will see some correlated activity in both the neigbboring areas, and main(cid:173)\ntain connections to both ocular dominance columns (figure 3b). \n\nThe lateral connection patterns shown above closely match  observations in the primary vi(cid:173)\nsual cortex.  Lowel and Singer (1992) observed that when between-eye correlations are abol(cid:173)\nished in kittens by surgically induced strabismus,  long-range lateral connections primarily \nlink areas of the same ocular dominance.  However, binocular neurons, located between ocu(cid:173)\nlar dominance columns, retained connections to both eye regions.  The receptive field model \nconfinns that such patterned lateral connections develop based on correlated neuronal activity, \nand demonstrates that they can self-organize simultaneously with ocular dominance columns. \nThe model also predicts that the long-range connections have an inhibitory function. \n\n4  Discussion \n\nIn LISSOM,  evolving lateral interactions and dynamic activity patterns are explicitly mod(cid:173)\neled.  Therefore,  LISSOM  has  several  novel  properties  that  set  it  apart  from  other  self(cid:173)\norganizing models of the cortex. \n\nPrevious models (e.g.  Goodhill1993; Milleret al.1989; Obermayer et al.1992; von der Mals(cid:173)\nburg 1973) have concentrated only on forming ordered topographic maps where clusters of \nadjacent neurons assume similar response properties such as ocular dominance or orientation \npreference.  The lateral connections in LISSOM, in addition, adapt to encode correlations be-\n\n\fOcular Dominance and Patterned Lateral Connections \n\n115 \n\ntween the responses. 2  This property can be potentially very useful in models of cortical func(cid:173)\ntion. While afferent connections learn to detect the significant features in the input space (such \nas ocularity or orientation), the lateral connections can learn correlations between these fea(cid:173)\ntures (such as Gestalt principles), and thereby form a basis for feature grouping. \n\nAs an illustration, consider a single spot of light presented to the left eye.  The spot causes dis(cid:173)\njoint activity patterns in the left-eye-dominant patches.  How can these multiple activity pat(cid:173)\nterns be recognized as representing the same spatially coherent entity? As proposed by Singer \net al. (1990), the long-range lateral connections between similar ocular dominance columns \ncould synchronize cortical activity,  and form a coherently firing assembly  of neurons.  The \nspatial coherence of the spot will then be represented by temporal coherence of neural activ(cid:173)\nity.  LISSOM can be potentially extended to model such feature binding. \n\nEven after the network has self-organized, the lateral and afferent connections remain plastic \nand in a continuously-adapting dynamic equilibrium with the input. Therefore, the receptive \nfield properties of neurons can dynamically readapt when the activity correlations in the net(cid:173)\nwork are forced  to change.  For example,  when a small area of the cortex is set inactive (or \nlesioned), the sharply-tuned afferent weight profiles of the neurons surrounding that region \nexpand in size, and neurons begin to respond to the stimuli that previously activated only the \nlesioned area (Sirosh and Miikkulainen 1994b, 1994c). This expansion of receptive fields is \nreversible,  and when  the lesion is repaired, neurons return to  their original tuning.  Similar \nchanges occur in response to retinal lesions as well.  Such dynamic expansions of receptive \nfields have been observed in the visual cortex (Pettet and Gilbert 1992). The LISSOM model \ndemonstrates that such plasticity is a consequence of the same self-organizing mechanisms \nthat drive the development of cortical maps. \n\n5  Conclusion \nThe LISSOM model shows how a single local and unsupervised self-organizing process can \nbe responsible for the development of both afferent and lateral connection structures in the pri(cid:173)\nmary visual cortex. It suggests that this same developmental mechanism also encodes higher(cid:173)\norder visual information such as feature correlations into the lateral connections.  The model \nforms a framework for future computational study of cortical reorganization and plasticity, as \nwell as dynamic perceptual processes such as feature grouping and binding. \n\nAcknowledgments \nThis  research  was  supported  in  part  by  National  Science  Foundation  under  grant  #IRI-\n9309273. Computer time for the simulations was provided by the Pittsburgh Supercomputing \nCenter under grants IRI930005P and TRA940029P. \nReferences \n\nBurkhalter,  A.,  Bernardo, K. L., and Charles,  V.  (1993).  Development of local circuits in \n\nhuman visual cortex.  Journalo/Neuroscience,  13:1916-1931. \n\nGilbert,  C.  D.,  and Wiesel,  T.  N.  (1989).  Columnar specificity of intrinsic horizontal and \ncorticocortical connections in cat visual cortex.  Journal 0/ Neuroscience, 9:2432-2442. \n\n2Tbe idea was conceived by von der Malsburg and Singer (1988), but not modeled. \n\n\f116 \n\nJoseph  Sirosh,  Risto  Miikkulainen \n\nGoodhill, G. (1993). Topography and ocular dominance:  a model exploring positive correla(cid:173)\n\ntions.  Biological Cybernetics, 69:109-118. \n\nKatz, L. C., and Callaway, E. M. (1992). Development of local circuits in mammalian visual \n\ncortex.  Annual Review o/Neuroscience,  15:31-56. \n\nKohonen, T.  (1982). Self-organized formation of topologically correct feature maps.  Biolog(cid:173)\n\nical Cybernetics, 43:59-69. \n\nLowel, S.  (1994).  Ocular dominance column development:  Strabismus changes the spacing \nof adjacent columns in cat visual cortex.  Journal 0/ Neuroscience,  14(12):7451-7468. \nLowel, S., and Singer, W.  (1992).  Selection of intrinsic horizontal connections in the visual \n\ncortex by correlated neuronal activity.  Science,  255:209-212. \n\nMalach,  R.,  Amir,  Y.,  Harel,  M.,  and Grinvald, A  (1993).  Relationship between  intrinsic \nconnections and functional architecture revealed by optical imaging and in vivo targeted \nbiocytin injections in the primate striate cortex.  Proceedings o/the National Academy \no/Sciences,  USA,90:10469-10473. \n\nMiller, K. D.  (1994).  A model for the development of simple cell receptive fields  and the \n\nordered arrangement of orientation columns through activity-dependent competition be(cid:173)\ntween on- and off-center inputs.  Journalo/Neuroscience,  14:409-441. \n\nMiller, K. D., Keller, 1. B., and Stryker, M. P. (1989). Ocular dominance column development: \n\nAnalysis and simulation.  Science, 245:605-615. \n\nObermayer, K., Blasdel, G. G., and Schulten, K. J. (1992).  Statistical-mechanical analysis of \nself-organization and pattern formation during the development of visual maps. Physical \nReview A, 45:7568-7589. \n\nPettet, M. W.,  and Gilbert, C. D.  (1992).  Dynamic changes in receptive-field size in cat pri(cid:173)\nmary visual cortex.  Proceedings o/the NationalAcademy 0/ Sciences,  USA,89:8366-\n8370. \n\nSinger, W.,  Gray, C., Engel, A, Konig, P.,  Artola, A, and Bracher, S.  (1990).  Formation of \ncortical cell assemblies.  In Cold Spring Harbor Symposia on Quantitative Biology,  Vol. \nLV, 939-952. Cold Spring Harbor, NY:  Cold Spring Harbor Laboratory. \n\nSirosh, J., and Miikkulainen, R. (1993). How lateral interaction develops in a self-organizing \nfeature map.  In Proceedings o/the IEEE International Conference on Neural Networks \n(San Francisco,  CA),  1360--1365. Piscataway, NJ: IEEE. \n\nSirosh, J., and Miikkulainen, R. (1994a). Cooperative self-organization of afferent and lateral \n\nconnections in cortical maps.  Biological Cybernetics, 71(1):66--78. \n\nSirosh, 1., and Miikkulainen, R. (1994b). Modeling cortical plasticity based on adapting lat(cid:173)\neral interaction. In The Neurobiologyo/Computation: Proceedings o/the Annual Com(cid:173)\nputationalNeuroscience Meeting. Dordrecht; Boston:  Kluwer.  In Press. \n\nSirosh, J., and Miikkulainen, R. (1994c). A neural network model oftopographic reorganiza(cid:173)\n\ntion following cortical lesions. In Proceedings o/the World Congress on Computational \nMediCine,  Public Health and BioteChnology (Austin, TX). World Scientific.  In Press. \n\nvon  der Malsburg,  C.  (1973).  Self-organization of orientation-sensitive cells  in the striate \n\ncortex.  Kybernetik,  15:85-100. \n\nvon der Malsburg, C., and Singer, W.  (1988).  Principles of cortical network organization.  In \nRakic, P.,  and Singer, W., editors, Neurobiology 0/ Neocortex, 69-99. New York:  Wiley. \n\n\f", "award": [], "sourceid": 1013, "authors": [{"given_name": "Joseph", "family_name": "Sirosh", "institution": null}, {"given_name": "Risto", "family_name": "Miikkulainen", "institution": null}]}