{"title": "Generating velocity tuning by asymmetric recurrent connections", "book": "Advances in Neural Information Processing Systems", "page_first": 325, "page_last": 332, "abstract": null, "full_text": "Generating velocity tuning by asymmetric\n\nrecurrent connections\n\n Dept. of Brain and Cognitive Sciences and CBCL\n\nXiaohui Xie and Martin A. Giese\u0002\u0001\nMassachusetts Institute of Technology\n\n72076 T\u00a8ubingen, Germany\n\nE-mail: \u0004 xhxie|giese\u0005 @mit.edu\n\nCambridge, MA 02139\n\n\u0003 Dept. for Cognitive Neurology,\nUniversity Clinic T\u00a8ubingen\n\nMax-Planck-Institute for Biological Cybernetics\n\nAbstract\n\nAsymmetric lateral connections are one possible mechanism that can ac-\ncount for the direction selectivity of cortical neurons. We present a math-\nematical analysis for a class of these models. Contrasting with earlier\ntheoretical work that has relied on methods from linear systems theory,\nwe study the network\u2019s nonlinear dynamic properties that arise when the\nthreshold nonlinearity of the neurons is taken into account. We show\nthat such networks have stimulus-locked traveling pulse solutions that\nare appropriate for modeling the responses of direction selective cortical\nneurons. In addition, our analysis shows that outside a certain regime\nof stimulus speeds the stability of this solutions breaks down giving rise\nto another class of solutions that are characterized by speci\ufb01c spatio-\ntemporal periodicity. This predicts that if direction selectivity in the cor-\ntex is mainly achieved by asymmetric lateral connections lurching activ-\nity waves might be observable in ensembles of direction selective cortical\nneurons within appropriate regimes of the stimulus speed.\n\n1 Introduction\n\nClassical models for the direction selectivity in the primary visual cortex have assumed\nfeed-forward mechanisms, like multiplication or gating of afferent thalamo-cortical inputs\n(e.g. [1, 2, 3]), or linear spatio-temporal \ufb01ltering followed by a nonlinear operation (e.g.\n[4, 5]). The existence of strong lateral connectivity has motivated modeling studies, which\nhave shown that the properties of direction selective cortical neurons can also be accurately\nreproduced by recurrent neural network models with asymmetric lateral excitatory or in-\nhibitory connections [6, 7]. Since these biophysically detailed models are not accessible\nfor mathematical analysis, more simpli\ufb01ed models appropriate for a mathematical analysis\nhave been proposed. Such analysis was based on methods from linear systems theory by\nneglecting the nonlinear properties of the neurons [6, 8, 9]. The nonlinear dynamic phe-\nnomena resulting from the interplay between the recurrent connectivity and the nonlinear\n\n\u0003\n\fthreshold characteristics of the neurons have not been tractable in this theoretical frame-\nwork.\n\nIn this paper we present a mathematical analysis that takes the nonlinear behavior of the\nindividual neurons into account. We present the result of the analysis of such networks\nfor two types of threshold nonlinearities, for which closed-form analytical solutions of the\nnetwork dynamics can be derived. We show that such nonlinear networks have a class of\nform-stable solutions, in the following signi\ufb01ed as stimulus-locked traveling pulses, which\nare suitable for modeling the activity of direction selective neurons. Contrary to networks\nwith linear neurons, the stability of the traveling pulse solutions in the nonlinear network\ncan break down giving raise to another class of solutions (lurching activity waves) that is\ncharacterized by spatio-temporal periodicity. Our mathematical analysis and simulations\nshowed that recurrent models with biologically realistic degrees of direction selectivity\ntypically also show transitions between traveling pulse and lurching solutions.\n\n2 Basic model\n\n\u0001\n\u0002\u0005\u0004\t\u0006\t\b\n\nis described by:\n\n\u0001\u0003\u0002\u001b\u001a\n\u0004\u0007\u0006\t\b\t\b\t\u001f \u0002\u001b\u001a\n\n\u0001\u0003\u0002#\u0004\t\u0006\t\b%$\n\n\u000e\"!\n\nDynamic neural \ufb01elds have been proposed to model the average behavior of a large ensem-\n\nThis dynamics is essentially a leaky integrator with a total input on the right hand side,\n\nrecurrent contributions from other laterally connected neurons. The interaction kernel\n\nmation of biophysically discrete neuronal dynamics it is in some cases possible to treat the\nnonlinear neural dynamics analytically.\n\n\u0001\u0003\u0002\u0005\u0004\u0007\u0006\t\b characterizes the\nbles of neurons [10, 11, 12]. The scalar neural activity distribution\naverage activity at time\u0006 of an ensemble of functionally similar neurons that code for the\nposition\u0002\n, where\u0002 can be any abstract stimulus parameter. By the continuous approxi-\n\u0001\n\u0002\u0005\u0004\u0007\u0006\t\b\nThe \ufb01eld dynamics of neural activation variable\n\u0001\u0003\u0002\u0005\u0004\u0007\u0006\t\b\r\u000f\u0011\u0010\u0013\u0012\u0015\u0014\u0016\u0001\u0003\u0002\u0018\u0017\u0019\u0002\u001b\u001a\u001c\b\t\u001d\u001e\u0001\n\u000b\r\f\n\u0001\n\u0002\u0005\u0004\u0007\u0006\t\b and a feedback term that integrates the\nwhich includes a feedfoward input term !\n\u0014\u0016\u0001\n\u0002&\u0017'\u0002\n\b characterizes the average synaptic connection strength between the neurons\n\u001a and the neurons coding position\u0002\ncoding position\u0002\nWith a moving stimulus at constant velocity( , it is often convenient to transform the static\n\u0006 . Under the new frame,\ncoordinate to the moving frame by changing variable )\n\u0006.\u0004\t\u0006\t\b . The dynamics for,\n\u0001\n\u0002-\u0017\nthe stimulus is stationary. Let,\n\u0004\t\u0006\t\b\n\u001a1\u0004\t\u0006\t\b\u0007\b2\u001f\n\u0004\t\u0006\t\b\r\u000f/\u0010\u0013\u00120\u0014\u0016\u0001\n\b\t\u001d\u001e\u0001\n\u0014\u0016\u0001\n\u000e\"!\n\nis the activation function of the\nneurons. This function is nonlinear and monotonically increasing, and introduces the non-\nlinearity that makes it dif\ufb01cult to analyze the network dynamics.\n\n\b corresponds to a traveling pulse solution with velocity(\n\nin the original static coor-\ndinate. Therefore the traveling pulse solution driven by the moving stimulus can be found\nby solving Eq. (3), and the stability of the traveling pulse can be studied by perturbing the\nstationary solution in Eq. (2).\n\n,43\n\nA stationary solution in the moving frame has to satisfy the following equation:\n\n\u000f*\u0002+\u0017\n\u001a\u001c\b\t\u001d\u001e\u0001\n\nreads\n\n\u000e\"!\n\n\b.$\n\n\u0004\u0007\u0006\t\b\r\u000f\n\n\b\r\u000f\n\n\b\u0007\b2\u001f\n\n\b.$\n\n(1)\n\n(2)\n\n(3)\n\n.\n\n\u0004\t\u0006\t\b\n\n,43\n\nThe neural \ufb01eld dynamics Eq. (2) is a nonlinear integro-differential equation.\nIn most\ncases an analytic treatment of such equations is impossible. In this paper, we consider\ntwo biologically inspired special cases, which can be analytically solved. For this purpose\nwe consider only one-dimensional neural \ufb01elds and assume that the nonlinear activation\n\nis either a step function or a linear threshold function.\n\nfunction\u001d\n\n\n\f\n\u0006\n\u000e\n\n\n\u001a\n\u001d\n(\n\u0001\n)\n\n(\n\u000b\n\f\n,\n\u0001\n)\n\f\n\u0006\n\u0017\n\u000b\n(\n\f\n,\n\u0001\n)\n\f\n)\n\u000e\n,\n\u0001\n)\n)\n\u0017\n)\n,\n\u0001\n)\n)\n\u001a\n\u0001\n)\n\u0017\n\u000b\n(\n\u001f\n\u0001\n)\n\b\n\u001f\n)\n\u000e\n,\n3\n\u0001\n)\n\u0010\n\u0012\n)\n\u0017\n)\n\u001a\n,\n3\n\u0001\n)\n\u001a\n)\n\u001a\n\u0001\n)\n\u0001\n)\n\f3 Step activation function\n\n\u000f\u0003\u0002\n\nobeys the ordinary differential equation\n\nzero otherwise. This form of activation function approximates activities of neurons, which,\nby saturation, are either active or inactive. For the one-dimensional case, we assume that\n\n\u0001\u0004\n\b where \u0002\n\u0001\u0004\n\u000f\u0006\u0005 when \b\u0007\n\t and\nWe \ufb01rst consider step activation function\u001d\u001e\u0001\u0001\u0013\b\n\b\u000b\u0007\f\t )exists that is located between the\nonly a single stationary excited regime with (,\n\b . Only neurons inside this regime contribute to the integral, and accordingly\npoints\u0001\n\b of the stationary solution\nEq. (3) can be simpli\ufb01ed following [11]. The spatial shape,\n\u0017\u000e\r\n\u000f\f\r\n\u0001\u0004\n\b\u0010\u000f\u0012\u0011\u0014\u0013\n\u0001\u0003\u0002\u001b\b\t\u001f \u0002\nwhere \r\nthe boundaries)\n and)\n\u0003 as \ufb01xed parameters, and solving Eq. (4).\nTo facilitate notation we de\ufb01ne an integral operator \u0016 with parameter \u0017\u0019\u0018\n\u001f(#\n\u0017\u000b2\nwhere \ntwo functions5\n\u0001\u0001\u0013\b\n\nThe solution of Eq. (4) can be written with these functions in the form\n\n. The solution of the above equation can be found by treating\n\n\b.\u0004\n\u000f\f\t as\n\notherwise. Using this operator we de\ufb01ne the\n\nfor \u001743\n\u0016\u001b\u001a\n\n\u0013*),+.-\u0001/10\n\n\u0001\u0001\u0013\b\r\u000f\n\n\u0017 \u001f\n\u00012\u0017\n\n\u0001\u0001\u0013\b\u001e\u001d\n\u0016\u001b\u001a\n\t and \n\b6\u001d\n\u0001\u0004\n((\u001f$7\n\n\u000f/\u0010\n\n\b&%('\n\n(9\u001f\u00017\n\n\u0016\u001b\u001a\n\n\b%$\n\nand\n\n\u0001\t\u0017\n\n\u0001$#\n\n\u0013\"!\n\n(4)\n\n(5)\n\n\b\u001e\u0017\n\n\b\r\u000f\n\n(7)\n(8)\n\n\b\u001e\u0017\n\u000f4\t must be satis\ufb01ed, leading to the transcen-\n\ndent equation system\n\n and)\n\nThe linearized perturbation dynamics reads\n\n\u0003 can be determined.\n\n3.1 Stability of the traveling pulse solution\n\nThe stability of the traveling pulse solution can be analyzed by perturbing the stationary\n\nFor the boundary points,\nfrom which)\nsolution in the moving coordinate system. Let : ,\n\u0014\u0016\u0001\n\u0014\u0016\u0001\nwhere:\n\u0004\t\u0006\t\b4\u000f\u0006\t . However, :\nthe stationary values of)\n: ,\n\u0004\t\u0006\t\b\nSince,\n\n\b .\n: ,\n\u000b\r\f\n\u000f=\u0005 \u00041> ) are the perturbations of the boundary points of the exited regime from\n\u0004\u0007\u0006\t\b\n\u0004\u0007\u0006\t\b\u0015\u000f\n\u0004\t\u0006\t\b\n) . Substituting this back into the perturbed dynamics, we have\n: ,\n\n)*; (<\n\u0004\t\u0006\t\b , and the dependence can be found by noting that\n\n\u0004\u0007\u0006\t\b\nto the \ufb01rst order we have :\n\u0004\t\u0006\t\b\r\u000f\n: ,\n\n\u0004\u0007\u0006\t\b be a small perturbation of,\n\n: ,\n: ,\n; with,\n\u0004\t\u0006\t\b\n\n\u000f\f\t\n; , where ?\n\u0004\t\u0006\t\b%$\n\nis not independent of\n\n: ,\n\u0014\u0016\u0001\n\n\u0004\t\u0006\t\b\r\u000f\n\n7@?\n: ,\n\n\u0014\u0016\u0001\n\n\u0004\u0007\u0006\t\b\n\n\u0004\t\u0006\t\b\n\n\u000b\r\f\n\n: ,\n\n: ,\n\n\u0001\u0001\u0013\b\u001e\u001d\n\b%$\n\n\b.\u0004\n\n(6)\n\n(9)\n\n\u0001\u0004\t\n\u0001\u0001\t\n\n: ,\n\n\b\n3\n\u0001\n)\n)\n3\n\n\u0004\n)\n3\n\u0003\n3\n\u0001\n)\n\u0017\n\u000b\n(\n\u001f\n,\n3\n\u0001\n)\n\b\n\u001f\n)\n\u000e\n,\n3\n\u0001\n)\n\b\n\u0001\n)\n\u0017\n)\n3\n\n\b\n\u0001\n)\n\u0017\n)\n3\n\u0003\n\b\n\u000e\n!\n\u0001\n)\n\u0015\n\u0014\n3\n3\n\u001c\n\u0013\n\u001c\n\u0004\n\u0015\n\u000f\n\u0015\n\u000f\n\u000e\n2\n\u000f\n\n\u000b\n\u000b\n(\n\b\n8\n!\n\u000b\n\u000b\n(\n,\n3\n\u0001\n)\n5\n\u0001\n)\n\u0017\n)\n3\n\n5\n\u0001\n)\n\u0017\n)\n3\n\u0003\n\b\n\u000e\n8\n\u0001\n)\n3\n\u0001\n)\n3\n\n\b\n\u000f\n,\n3\n\u0001\n)\n3\n\u0003\n\b\n\u0017\n5\n\b\n\u000e\n5\n\u0001\n)\n3\n\n\u0017\n)\n3\n\u0003\n\b\n\u000f\n8\n\u0001\n)\n3\n\n\b\n5\n5\n\u0001\n)\n3\n\u0003\n\u0017\n)\n3\n\n\b\n\u000f\n8\n\u0001\n)\n3\n\u0003\n3\n3\n\u0001\n)\n3\n\u0001\n)\n\f\n\u0006\n\u0017\n\u000b\n(\n\f\n\f\n)\n\u000e\n\u0001\n)\n\u0017\n)\n\u0017\n)\n3\n\n\b\n:\n)\n\n\u000e\n)\n\u0017\n)\n3\n\u0003\n\b\n:\n)\n\u0003\n\u0004\n3\n\u0001\n)\n3\n;\n\u000e\n:\n)\n;\n)\n;\n\u0001\n)\n,\n\u0001\n)\n;\n\u000f\n,\n\u0001\n)\n3\n;\n\u000e\n:\n)\n;\n\u000f\n,\n\u0001\n)\n3\n;\n\u000e\n\f\n,\n\u0001\n)\n3\n;\n\f\n)\n:\n)\n;\n\u000e\n\u0016\n\u0001\n:\n)\n\u0003\n;\n\b\n$\n\u0001\n)\n3\n;\n\u0001\n)\n3\n;\n)\n;\n\u000f\n\u0017\n\u0001\n)\n3\n;\n3\n3\n;\n\u000f\n\u001f\n,\n3\n\u0001\n)\n;\n\b\n7\n\u001f\n\f\n\u0006\n\u0017\n\u000b\n(\n\f\n\f\n)\n\u000e\n\u0001\n)\n)\n\u0017\n)\n3\n\n\b\n?\n3\n\n\u0001\n)\n3\n\n\u0017\n)\n\u0017\n)\n3\n\u0003\n\b\n?\n3\n\u0003\n\u0001\n)\n3\n\u0003\n\f(10)\n\n\u0001\u0001\n\n\u0001\u0001\t\n\n\u0001\t\b\n\n\b.\u0004\n\n\u0016\u001b\u001a\n\nreads\n\nis de\ufb01ned as\n\n\u000f\u0003%\u0001\u0003\u0002\u0005\u0004\n\nWe use the following function\n\ninto the above dynamics. After some\n\n\u0004\u0007\u0006\t\b\nSubstitute solution of the form : ,\ncalculation, the eigenvalue equation for\u0006\n\u0001\u0004\t\n\b\u001e\u0017\nwhere function\u0007\n\u00012\u0017\n\b6\u001d\n\u0014\u0016\u0001\u0004\n\b\r\u000f\nFrom the transcendent Eq. (10),\u0006 can be found. The traveling pulse solution is asymptoti-\ncally stable only if the real parts of all eigenvalues\u0006 are negative.\n\u000e\u0019\u0010\u001a\u0012#\u00012\u0017\u0014\u0013\n\n3.2 Simulation results of step activation function model\n\n\u000f\u000b\n\r\f\u000f\u000e\u0011\u0010\r\u0012\u0005\u0001\t\u0017\u0014\u0013\u0015\f\u0017\u0016\n\nas an example interaction kernel, numerically simulate the dynamics and compare the sim-\nulation results with the above mathematical analysis. The stimulus used is a moving bar\nwith constant width and amplitude. The results are shown in the left (a-e) panels of Fig. (1).\nPanel (a) shows the speed tuning curve plotted as the dependence of the peak activity of the\n. The solid lines indicate the results\nfrom the numerical simulation and the dotted lines represent results from the analytical so-\nlution. Panel (b) shows the maximum real part of the eigenvalues obtained from Eq. (10).\n\ntraveling pulse as function of the stimulus velocity(\nFor small and large stimulus velocities maximum of the real parts of \u0006 becomes positive\n\nindicating a loss of stability of the form-stable solution. To verify this result we calculated\nthe variability of the peak activity over time in simulation. Panel (c) shows the average\nvariability as function of the stimulus velocity. At the velocities for which the eigenvalues\nindicate a loss of stability the variability of the amplitudes suddenly increases, consistent\nwith our interpretation as a loss of the form stability of the solution.\n\n\u0017\u0018\n\n\u0001\u0003\u0002\u001b\b\n\nAn interesting observation is illustrated in panels (d) and (e) that show a color-coded plot of\nthe space-time evolution of the activity. Panel (e) shows the propagation of the form-stable\ntraveling pulse. Panel (d) shows the solution that arises when stability is lost. This solution\nis characterized by a spatio-temporal periodicity that is de\ufb01ned in the moving coordinate\n\nsystem by,\n\n\u0001\u001c\u001b\n\n#\u001e\u001d\n\n\u0004\u0007\u0006\n\n\u000e \u001f\"!\n\n\b4\u000f\n\n\u0001#\u001b\u001b\u0004\u0007\u0006\t\b , where\u001d\n\n\u0015 and!\n\non the network dynamics. Solutions of similar type have been described before in spiking\nnetworks [13].\n\n\u0015 are constants that depend\n\n4 Linear threshold activation function\n\nIn this case, the activation function is taken to be \u001d\u001e\u0001\u0004\n\nneurons typically operate far below the saturation level. The linear threshold activation\nfunction is thus more suitable to capture the properties of real neurons while still permitting\na relatively simple theoretical analysis. We consider a ring network with periodic boundary\nconditions. The dynamics is given by\n\n\u0005 . Cortical\n\n\u000f&%('\u0001\u0010\n\n\u0004\"\t\n\n\u001f#$\n\n\u0001\u001c)\n\n\u0004\t\u0006\t\b\n\n\u0001#)\n\n\u0004\t\u0006\t\b\n\n\u000f+*\n\u0010-,\n\n\u001f\u0015)\n>\u0017.\n\n\u0014\u0016\u0001\u001c)\u0016\u0017/) \u001a\n\n\u0001\u001c)\n\n\u001a1\u0004\u0007\u0006\t\b\n\n\u0001\u001c)\n\n\u0004\u0007\u0006\t\b10\n\n\u000e\"!\n\nThis network can be shown equivalent to the the standard one in Eq. (1) by changing vari-\nables and transforming stimulus. We chose this form because it simpli\ufb01es the mathematical\n\nlyze the network in the moving frame.\n\nanalysis of ring networks. Again, we consider a moving stimulus with velocity( and ana-\n\n(11)\n\n\u0001\n)\n\u0001\n)\n\b\n\u001a\n\u0007\n?\n3\n\n\u0001\n\u0005\n\u000e\n\u000b\n\u0006\n\b\n\u001f\n\u001a\n\u0007\n\b\n\u000e\n?\n3\n\u0003\n\u0001\n\u0005\n\u000e\n\u000b\n\u0006\n\b\n\u001f\n\u000f\n\u0007\n\u0001\n)\n3\n\n\u0017\n)\n3\n\u0003\n\u0004\n\u0006\n\b\n\u0007\n\u0001\n)\n3\n\u0003\n\u0017\n)\n3\n\n\u0004\n\u0006\n\b\n\u0007\n\u0004\n\u0006\n\u000b\n(\n7\n\u0001\n\u0005\n\u000e\n\u000b\n\u0006\n\b\n\u001f\n\u0001\n\u0005\n\u000e\n\u000b\n\u0006\n\b\n7\n\u000b\n(\n\b\n$\n\u0014\n\u0002\n\u0017\n\u0002\n\u0015\n\u0016\n\b\n;\n;\n\u0016\n\u0002\n\u0017\n\u0002\n\u0015\n\u0016\n\b\n\u000e\n\u0015\n\u0015\n,\n\b\n\u000f\n\u001a\n\n\u0004\n\n\u000b\n\f\n\f\n\u0006\n#\n\u000e\n#\n)\n,\n\u001a\n\b\n#\n$\n$\n\f4.1 General solutions and stability analysis\n\n\u0001#)\n\n\u0004\t\u0006\t\b\n\n\u0004\u0007\u0006\t\b\n\n\u0004\u0007\u0006\t\b\n\nBecause the activation function has linear threshold characteristics, inside the excited\n\n\t ) is positive the system is linear. One ap-\n\nregime for which the total input (\n\nbe written as:\n\n\u0001\u001c)\n#\u0007\u0006\n\nproach to solve this dynamics is therefore to \ufb01nd the solutions to the differential equation\nassuming the boundaries of the excited regime are given. The conditions at the bound-\naries lead to a set of self-consistent equations for the solutions to satisfy, from which the\nboundaries can be determined.\n\nBy denoting activities in moving coordinates as \n*1\u0010\n\u0001\u0003\u0006\t\b%\u0004\t)\n\n\u0006.\u0004\u0007\u0006\t\b\r\u000f\u0019#\n\u0001\u001c)\n\u0004\t\u0006\t\b , the dynamics can\n\u0001\u001c)\n\u0001\u001c)\n\u0004\u0007\u0006\t\b\u001e\u0001\u0004>\u0017.\n\u0004\t\u0006\t\b\n\u0001#)\n\u0014\u0016\u0001#)4\u0017\u0018)\n\u0002\u0001\n\u0001\n\u0006\t\b\t\b , we solve the dynamics by Fourier trans-\n\u0001\u001c)\nSupposing the excited regime is)\u0004\u0003\n\u0004\t.\u0005\b . Let\nforming the above equation in the spatial domain \u001a\n\u0014\u0016\u0001\u001c)\n>\u0017.#\b\n\b&%\n\u000f/\u0010\n\u0014\u000e\u0006\n;\t\b\u000b\n\r\f\n\b&%\n\u0001#)\n\u000f\u0011\u0010\n\u0001\u0004>\u0001.\u0005\b\n\f\u0016\u0015\n\b\u001a\u0019\n'&%\u001b\u0001\n\n\u0001\u001c)\n\u0001\u001c)\n\u0004\u0007\u0006\t\b&%\n>\u0017.\u0005\b\n\u0001\u0004>\u0001.\u0005\b\n$ is the frequency. The stationary solution in moving coordinates can\n\u0001\u001f\u001e\n(! \n\b . The components of the\nis de\ufb01ned as the diagonal matrix \n, and those of \u001c\nare \u0005\n and)\n\u0003 can be determined.\n\u0001\u001c)\n\u0005 \u00041> \b ,where\n\nStability of this traveling pulse solution can be analyzed by linear perturbation. Note that\nperturbed boundaries points do not contribute to the linearized perturbed dynamics since\nis the total input at the stationary solution of the\nmoving frame on right hand side of Eq. (11). Therefore, the linearized perturbation dy-\nnamics can be fully characterized by the perturbed Fourier modes with \ufb01xed boundaries.\nHence, the stability of the traveling pulse solution is determined by the eigenvalues of ma-\nis negative, then\n\n\u0001\n\u0006\t\b\r\u000f'\u0010\n\u0014\u0014\u0012\n\u0006\u0013\u0012\u0005\u000f\n\u0004\u000b\u001b\n\u000f4\t\nwhere\u0013\nwhere matrix \n#'\u0006\nvector \u001c\nare \u0005\nconditions, from which)\n\n. The above solution has to satisfy two boundary\n\n;\u000f\b\u000b\n\u0010\f\n;\t\b\n\nthen be written as\n\n\f\u0016\u0015\n\n\f\u0018\u0017\n\n\f\u0018\u0017\n\n\u0001$#\n\n\u0013!\u0006\n\n\b . If the largest real part of eigenvalues of (\n\n\u0001#)\n\ntrix (\n\n\u00174\u0001)\u001e\n\n(\u0010 \n\nthe stimulus locking traveling pulse is stable.\n\n4.2 Simpli\ufb01ed linear threshold network\n\n\u0001\u001c)\n\n(12)\n\nThe interaction kernel and feedforward input are assumed to have the following form:\n\nThe general solution introduced above requires the solution of an equation system.\nIn\npractice, the Fourier series have to be truncated in order to obtain a \ufb01nite number of Fourier\ncomponents at the expense of an approximation error. Next we consider a special simple\nmodel for which an exact solution can be found that contains only two Fourier components\n\nstability analysis is presented, that at the same time provides insight in some rather general\nproperties of linear threshold networks.\n\nand the input ! . For this model a closed form solution and\n\nfor the interaction kernel \u0014\n\u000f+*\n\b and a form-constant moving stimulus!\n\n\u0014\u0016\u0001\u001c)\ninteraction kernel\u0014\u0016\u0001\u001c)\n\nThis network was used by Hansel and Sompolinsky as model of cortical orientation se-\nlectivity [14]. However different from their network, we consider here an asymmetric\n\n-,\u0002.\u0010/\n\u0001\u001c)\u0015\u0017\n\n\u0001#)\n\u0006\t\b .\n\n\u000e10\n\n-,\u0002.\r/\n\n\b\r\u000f\n\n\u0001\u001c)\n\n\u0001\u001c)\n\n(13)\n\n\u0007\n\u0017\n(\n\u000b\n\f\n\f\n\u0006\n\n\u0017\n\u000b\n(\n\f\n\f\n)\n\n\u000e\n\n\u000f\n,\n)\n,\n\u001a\n\b\n\n\u001a\n\b\n)\n)\n\u001a\n\u000e\n8\n\b\n0\n$\n$\n\n\u0003\n\u0017\n.\n\u0005\n,\n)\n,\n\n\u0001\n)\n\n\u0001\n)\n\u0005\n,\n)\n,\n\u0001\n)\n\n\u0001\n)\n\u0011\n\u0005\n\u0010\n%\n;\n'\n\b\n\n)\n-\n\f\n)\n\n\u0001\n)\n\u0005\n!\n\u0006\n8\n\n\f\n)\n\n\u0001\n)\n$\n\u0006\n\u0005\n\u0004\n$\n$\n\u001c\n\u001d\n3\n\u000f\n\u000e\n<\n\u000b\n\u0017\n\u0011\n\b\n)\n\n\u001c\n\"\n\u0004\n\u000f\n\u001d\n\"\n!\n\u0006\n:\n)\n;\n\n3\n;\n\b\n\u000f\n\t\n\u0001\n<\n\u000f\n3\n\b\n\u000f\n\u000e\n<\n\u000b\n\u0017\n\u0011\n\b\n\u0015\n\u000e\n*\n\b\n!\n\u0011\n\u0015\n\u0017\n\u0011\n\b\n(\n\fSince the interaction kernel\u0014\n>\u0001.\n\n\u0001\n\u0006\t\b\r\u000f\n\nand input! only involve \ufb01rst two Fourier components, the\n\u0001\u001c)\n\u0004\u0007\u0006\t\b\n\n)\u0002\u0001 -\n\n\u0001\u0003\u0006\t\b\n\n(14)\n\n\u0001\u001c)\n\n\u0004\t\u0006\t\b\n\n>\u0001.\n\ndynamics can be fully determined in terms of its order parameters de\ufb01ned by\n\n\u0001\u0003\u0006\t\b\n\nis to restrict \n\nto being real. In terms of these two order param-\nwhere phase variable \u0003\neters plus the phase variable, the stimulus-locked traveling pulse solution and its stability\nconditions can be expressed analytically. Due to space limitation, the detailed derivations\nare omitted here. We show the theoretical results in right \ufb01ve panels of Fig. (1) and compare\nthem with numerical simulations.\n\n\u0015 and \n\nSimilar to the results of step function model, panel (A) shows the speed tuning curve plotted\nas values of order parameters \n(B) shows the largest real part of the eigenvalues of a stability matrix that can be obtained by\nlinearizing the order parameter dynamics around the stationary solution. Panel (C) shows\nthe average variations as function of the stimulus velocity. The space-time evolution of\nthe form-stable traveling pulse is shown in panel (E); the form-unstable lurching wave is\nshown in panel (D). Thus we found that lurching wave solution type arises very robustly for\nboth types of threshold functions when the network achieved substantial direction selective\nbehavior.\n\n as function of different stimulus velocities( . Panel\n\n5 Conclusion\n\nWe have presented different methods for an analysis of the nonlinear dynamics of simple\nrecurrent neural models for the direction selectivity of cortical neurons. Compared to ear-\nlier works, we have taken into account the essentially nonlinear effects that are introduced\nby the nonlinear threshold characteristics of the cortical neurons. The key result of our\nwork is that such networks have a class of form-stable traveling pulse solutions that behave\nsimilar as the solutions of linear spatio-temporal \ufb01ltering models within a certain regime\nof stimulus speeds. By the essential nonlinearity of the network, however, bifurcations can\narise for which the traveling pulse solutions become unstable. We observed that in this\ncase a new class of spatio-temporally periodic solutions (\u201dlurching activity waves\u201d) arises.\nSince we found this solution type very frequently for networks with substantial direction\nselectivity our analysis predicts that such \u201dlurching behavior\u201d might be observable in visual\ncortex areas if, in fact, the direction selectivity is essentially based on asymmetric lateral\nconnectivity.\n\nAcknowledgments\n\nWe acknowledge helpful discussions with H.S. Seung and T. Poggio.\n\nReferences\n\n[1] C Koch and T Poggio. The synaptic veto mechanism: does it underlie direction\nand orientation selectivity in the visual cortex. In D Rose and V G Dobson, editors,\nModels of the Visual Cortex, pages 15\u201334. John Wiley, 1989.\n\n[2] J.P. van Santen and G. Sperling. Elaborated reichardt detectors. J Opt Soc Am A,\n\n256:300\u201321, 1985.\n\n[3] W. Reichardt. A principle for the evaluation of sensory information by the central\n\nnervous system, 1961.\n\n[4] E. H. Adelson and J. R. Bergen. Spatiotemporal energy models for the perception of\n\nmotion. J Opt Soc Am A, 256:284\u201399, 1985.\n\n\n\u0015\n\u0010\n,\n)\n,\n\u001f\n)\n\u001a\n#\n\u001a\n\n\n\u000f\n\u0010\n,\n)\n,\n\u001f\n)\n\u001a\n#\n\u001a\n%\n;\n'\n\f\n\n\ftheory \nsimulation\n\n-30\n\n-20\n\n10\n\n-30\n\n-20\n\n10\n\n-30\n\n-20\n\n10\n\nVelocity\n\ne\n\na\n\n3\n\n2\n\n1\n\ny\nt\ni\nv\ni\nt\nc\nA\n \nk\na\ne\nP\n\n0\n-40\n2\n\n)\n\n(l\nl\na\ne\nR\n\n0\n\nb\n\n-2\n-40\n0.2\n\n0.1\n\n0\n-40\n\ne\nc\nn\na\ni\nr\na\nv\n \nk\na\ne\nP\n\nE\nM\nT\n\nI\n\nc\n\nd\n\ny\nt\ni\nv\ni\nt\nc\nA\n\n(cid:13) A\n\nr\n0\nr\n1\n\n0.03\n\n0.02\n\n0.01\n\n0\n-50\n\n0.5\n\n0\n\n0\n\n0\n\n0\n\n10\n\n10\n\n10\n\ni\n\ng\nE\n\n \nl\n\na\ne\nR\n\nB\n\n-0.5\n-50\n\nx 10-3\n\n3\n\n2\n\n1\n\nn\no\n\ni\nt\n\na\ni\nr\na\nV\n\nC\n\n0\n-50\n\nD\n\nE\nM\nT\n\nI\n\n100\n\n100\n\n100\n\n0\n\n0\n\n0\n\n50\n\n50\n\n50\n\nVelocity\n\nE\n\nSPACE\n\nSPACE\n\nFigure 1: Traveling pulse solution and its stability in two classes of models. In the left side\nshown is the step activation function model, while the linear threshold model is drawn in\nthe right. Panel (a) and (A) show the velocity tuning curves of the traveling pulse in terms\nof its peak activity in (a) or order parameters in (A). The solid lines indicate the results\nfrom calculation, and the dotted lines represents the results from simulaion. Panel (b) and\n(B) plot the largest real parts of eigenvalues of a stability matrix obtained from perturbed\nlinear dynamics around the stationary solution. Outside certain range of stimulus velocities\nthe largest real part of the eigenvalues become positive indicating a loss of stability of the\nform-stable solution. Panel (c) and (C) plots the average variations of peak activity, and\norder parameters \nlation. A nonzero variance signi\ufb01es a loss of stability for traveling pulse solutions, which\nis consistent with eigenvalue analysis in Panel (b) and (B). A color coded plot of spatial-\nin (D)\nand (E). Panel (e) and (E) show the propagation of the form-stable peak over time; panel (d)\nand (D) show the lurching activity wave that arises when stability is lost. The interaction\n\n\u0015 (blue curve) and \n (green curve) respectively, over time during simu-\n\u0001\u0003\u0002#\u0004\t\u0006\t\b\n\u0001\u0003\u0002#\u0004\t\u0006\t\b\nis shown in panels (d) and (e), and#\ntemporal evolution of the activity\n\u000f\u000b\n\n\u0002\u0016\u0017\n\u000e\u0011\u0010\r\u0012\u0005\u0001\t\u0017\u0014\u0013\n\f\u000f\u000e\u0011\u0010\r\u0012\u0005\u00012\u0017\u0014\u0013\nkernel used in step function model is\u0014\u0016\u0001\n\u0002\u001b\b\n\u0002\u0016\u0017\n\f\u0001\u0016\n\u000f\u0001\n\u0005 and\u0002\n\u0005 \u0004\n\u000f\u0001\u0003 . The stimulus is a moving bar\nwith \n\n\u000f\u0006> . Parameters used in linear threshold model are\nwith width\u001f\u0019\u000f\n\u0005*\t and amplitude\n\t\t and\u000b\n\u0005\u000b\n .\n\u0005\b\u0003\n\u0007 ,*\n\n\u0017\u0006\u0005\n\n , \u0011\n\n(cid:13)\n\u0002\n\u0015\n\u0016\n\b\n\u0017\n\n;\n;\n\u0016\n\u0002\n\u0015\n\u0016\n\b\n\f\n\u000f\n\n;\n\u0004\n\u0013\n\f\n\u000f\n\t\n$\n\u0002\n>\n\u0004\n\u0013\n;\n\u000f\n\t\n$\n\u0015\n\u0004\n*\n\u0015\n\u000f\n$\n\n\u000f\n$\n\u0015\n\u000f\n\u0011\n\n\u000f\n\t\n$\n\u000f\n\t\n$\n\t\n\f[5] A. B. Watson and A. J. Ahumada. Model of human visual-motion sensing. J Opt Soc\n\nAm A, 256:322\u201341, 1985.\n\n[6] H. Suarez, C. Koch, and R. Douglas. Modeling direction selectivity of simple cells in\nstriate visual cortex within the framework of the canonical microcircuit. J Neurosci,\n15:6700\u201319, 1995.\n\n[7] R. Maex and G. A. Orban. Model circuit of spiking neurons generating directional\n\nselectivity in simple cells. J Neurophysiol, 75:1515\u201345, 1996.\n\n[8] P. Mineiro and D. Zipser. Analysis of direction selectivity arising from recurrent\n\ncortical interactions. Neural Comput, 10:353\u201371, 1998.\n\n[9] S. P. Sabatini and F. Solari. An architectural hypothesis for direction selectivity in the\nvisual cortex: the role of spatially asymmetric intracortical inhibition. Biol Cybern,\n80:171\u201383, 1999.\n\n[10] HR Wilson and JD Cowan. A mathematical theory of the functional dynamics of\n\ncortical and thalamic nervous tissue. Kybernetik, 13(2):55\u201380, 1973.\n\n[11] S Amari. Dynamics of pattern formation in lateral-inhibition type neural \ufb01elds. Biol\n\nCybern, 27(2):77\u201387, 1977.\n\n[12] E. Salinas and L.F. Abbott. A model of multiplicative neural responses in parietal\n\ncortex. Proc. Natl. Acad. Sci. USA, 93:11956\u201311961, 1996.\n\n[13] D. Golomb and G. B. Ermentrout. Effects of delay on the type and velocity of trav-\nelling pulses in neuronal networks with spatially decaying connectivity. Network,\n11:221\u201346, 2000.\n\n[14] David Hansel and Haim Sompolinsky. Modeling feature selectivity in local cortical\ncircuits. In C. Koch and I. Segev, editors, Methods in Neuronal Modeling, chapter 13,\npages 499\u2013567. MIT Press, Cambridge, Massachusetts, 1998.\n\n\f", "award": [], "sourceid": 1995, "authors": [{"given_name": "Xiaohui", "family_name": "Xie", "institution": null}, {"given_name": "Martin", "family_name": "Giese", "institution": null}]}