{"title": "Mathematical Analysis of Learning Behavior of Neuronal Models", "book": "Neural Information Processing Systems", "page_first": 164, "page_last": 173, "abstract": null, "full_text": "164 \n\nMATHEMATICAL ANALYSIS OF LEARNING BEHAVIOR \n\nOF NEURONAL MODELS \n\nBy \n\nJOHN Y. CHEUNG \nMASSOUD OMIDVAR \n\nSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE \n\nUNIVERSITY OF OKLAHOMA \n\nNORMAN, OK 73019 \n\nPresented to the IEEE Conference on \"Neural Information Processing Systems(cid:173)\nNatural and Synthetic,\" Denver, November ~12, 1987, and to be published in \nthe Collection of Papers from the IEEE Conference on NIPS. \n\nPlease address all further correspondence to: \n\nJohn Y. Cheung \nSchool of EECS \n202 W. Boyd, CEC 219 \nNorman, OK 73019 \n(405)325-4721 \n\nNovember, 1987 \n\n\u00a9 American Institute of Physics 1988 \n\n\f165 \n\nMATHEMATICAL ANALYSIS OF LEARNING BEHAVIOR \n\nOF NEURONAL MODELS \n\nJohn Y. Cheung and Massoud Omidvar \n\nSchool of Electrical Engineering \n\nand Computer Science \n\nABSTRACT \n\nIn this paper, we wish to analyze the convergence behavior of a number \nof neuronal plasticity models. Recent neurophysiological research suggests that \nthe neuronal behavior is adaptive. In particular, memory stored within a neuron \nis associated with the synaptic weights which are varied or adjusted to achieve \nlearning. A number of adaptive neuronal models have been proposed in the \nliterature. Three specific models will be analyzed in this paper, specifically the \nHebb model, the Sutton-Barto model, and the most recent trace model. In this \npaper we will examine the conditions for convergence, the position of conver(cid:173)\ngence and the rate at convergence, of these models as they applied to classical \nconditioning. Simulation results are also presented to verify the analysis. \n\nINTRODUCTION \n\nA number of static models to describe the behavior of a neuron have been \nin use in the past decades. More recently, research in neurophysiology suggests \nthat a static view may be insufficient. Rather, the parameters within a neuron \ntend to vary with past history to achieve learning. It was suggested that by \naltering the internal parameters, neurons may adapt themselves to repetitive \ninput stimuli and become conditioned. Learning thus occurs when the neurons \nare conditioned. To describe this behavior of neuronal plasticity, a number \nof models have been proposed. The earliest one may have been postulated \nby Hebb and more recently by Sutton and Barto 1. We will also introduce a \nnew model, the most recent trace (or MRT) model in this paper. The primary \nobjective of this paper, however, is to analyze the convergence behavior of these \nmodels during adaptation. \n\nThe general neuronal model used in this paper is shown in Figure 1. There \nare a number of neuronal inputs x,(t), i = 1, ... , N. Each input is scaled by \nthe corresponding synaptic weights w,(t), i = 1, ... , N. The weighted inputs \nare arithmetically summed. \n\nN \n\ny(t) = L x,(t)w,(t) - 9(t) \n\n,=1 \n\n(1) \n\nwhere 9(t) is taken to be zero. \n\n\f166 \n\nNeuronal inputs are assumed to take on numerical values ranging from zero \nto one inclusively. Synaptic weights are allowed to take on any reasonable values \nfor the purpose of this paper though in reality, the weights may very well be \nbounded. Since the relative magnitude of the weights and the neuronal inputs \nare not well defined at this point, we will not put a bound on the magnitude \nof the weights also. The neuronal output is normally the result of a sigmoidal \ntransformation. For simplicity, we will approximate this operation by a linear \ntransformation. \n\nSigmodial \nTransfonution \n\nneuronal \noutput \n\nH+-+y \n\nrilure 1. A leneral aeuronal .adel. \n\nFor convergence analysis, we will assume that there are only two neuronal \ninputs in the traditional classical conditioning environment for simplicity. Of \ncourse, the analysis techniques can be extended to any number of inputs. In \nclassical conditioning, the two inputs are the conditioned stimulus Xc (t) and \nthe unconditioned stimulus xu(t). \n\nTHE SUTTON-BARTO MODEL \n\nMore recently, Sutton and Barto 1 have proposed an adaptive model based \n\non both the signal trace x,(t) and the output trace y(t) as given below: \n\nw,(t + 1) =w,(t) + cx,(t)(y(t)) - y(t) \ny(t + 1) ={Jy(t) + (1 - {J)y(t) \nXi(t + 1) =axi(t) + Xi(t) \n\n(2a) \n(2b) \n(2c) \n\nwhere both a and {J are positive constants. \n\n\f167 \n\nCondition of Convergence \n\nIn order to simplify the analysis, we will choose Q = 0 and (3 = 0, i.e.: \n\nand \n\nIn other words, (2a) becomes: \n\n%,(t) = x,(t - 1) \n\ny(t) = y(t - 1) \n\nWi(t + 1) = Wi(t) + CXi(t)(y(t) - y(t - I)} \n\n(3) \n\nThe above assumption only serves to simplify the analysis and will not affect the \nconvergence conditions because the boundedness of %i(t) and y(t) only depends \non that for Xi(t) and y(t - 1) respectively. \n\nAs in the previous section, we recognize that (3) is a recurrence relation so \nconvergence can be checked by the ratio test. It is also possible to rewrite (3) \nin matrix format. Due to the recursion of the neuronal output in the equation, \nwe will include the neuronal output y(t) in the parameter vector also: \n\n(4) \n\nor \n\nTo show convergence, we need to set the magnitude of the determinant of \n\nA (S-B) to be less than unity. \n\nHence, the condition for convergence is: \n\n(5) \n\n(6) \n\nFrom (6), we can see that the adaptation constant must be chosen to be less \nthan the reciprocal of the Euclidean sum of energies of all the inputs. The \nsame techniques can be extended to any number of inputs. This can be proved \nmerely by following the same procedures outlined above. \n\nPosition At Convergence \n\n\f168 \n\nHaving proved convergence of the Sutton-Barto model equations of neu(cid:173)\n\nronal plasticity, we want to find out next at what location the system remains \nwhen converged. We have seen earlier that at convergence, the weights cease to \nchange and so does the neuronal output. We will denote this converged position \n\nas (W(S-B\u00bb- = W(S-B) (00). In other words: \n\n(7) \n\nSince any arbitrary parameter vector can always be decomposed into a weighted \nsum of the eigenvectors, i.e. \n\n(8) \n\nThe constants Ql, Q2, and Q3 can easily be found by inverting A(5-B). The \neigenvalues of A(5-B) can be shown to be 1, 1, and c(%j + %~}. When c is \nwithin the region of convergence, the magnitude of the third eigenvalue is less \nthan unity. That means that at convergence, there will be no contribution from \nthe third eigenvector. Hence, \n\n(9) \n\nFrom (9), we can predict precisely what the converged position would be given \nonly with the initial conditions. \n\nRate of Convergence \n\nWe have seen that when c is carefully chosen, the Sutton-Barto model will \nconverge and we have also derived an expression for the converged position. \nNext we want to find out how fast convergence can be attained. The rate \nof convergence is a measure of how fast the initial parameter approaches the \noptimal position. The asymptotic rate of convergence is2 : \n\n(10) \nis the spectral radius and is equalled to c(%~ + %~) in this \nwhere SeA (5-B\u00bb \ncase. This completes the convergence analysis on the Sutton-Barto model of \nneuronal plasticity. \n\nTHE MRT MODEL OF NEURONAL PLASTICITY \n\nThe most recent trace (MRT) model of neuronal plasticity 3 developed by \nthe authors can be considered as a cross between the Sutton-Barto model and \nthe Klopf's model \". The adaptation of the synaptic weights can he expressed \nas follows: \n\n(11) \n\n\f169 \n\nA comparison of (11) and the Sutton-Barto model in (3) ahOWl that the .cond \nterm on the right hand aide contains an extra factor, Wi(t), which iI used to \napeed up the convergence as ahoWD later. The output trace hu been replaced \nby If(t - 1), the most recent output, hence the name, the most recent trace \nmodel. The input trace is also replaced by the most recent input. \n\nCondition of Convergence \n\nWe can now proceed to analyze the condition of convergence for the MRT \nmodel. Due to the presence of the Wi(t) factor in the second term in (31), the \nratio test cannot be applied here. To analyze the convergence behavior further, \nlet us rewrite (11) in matrix format: \n\n0) ( WI(t) ) \n\nW2(t) \ny(t - 1) \n\no \no \n\nor \n\n(12) \n\nThe superscript T denotes the matrix transpose operation. The above equation \nis quadratic in W(MRT)(t). Complete convergence analysis of this equation is \nextremely difficult. \n\nIn order to understand the convergence behavior of (12), we note that \nthe dominant term that determines convergence mainly relates to the second \nquadratic term. Hence for convergence analysis only, we will ignore the first \nterm: \n\n(13) \nWe can readily see from above that the primary convergence factor is BT c. \nSince C is only dependent on %,(t), convergence can be obtained if the duration \nIt can be shown that the \nof the synaptic inputs being active is bounded. \ncondition of convergence is bounded by: \n\n(14) \n\n\f170 \n\nWe can readily see that the adaptation constant c can be chosen according \n\nto (14) to ensure convergence for t < T. \n\nSIMULATIONS \n\nTo verify the theoretical analysis of these three adaptive neuronal models \nbased on classical conditioning, these models have been simulated on the mM \n3081 mainframe using the FORTRAN language in single precision. Several test \nscenarios have been designed to compare the analytical predictions with actual \nsimulation results. \n\nTo verify the conditions for convergence, we will vary the value of the \nadaptation constant c. The conditioned and unconditioned stimuli were set \nto unity and the value of c varies between 0.1 to 1.0. For the Sutton-Barto \nmodel the simulation given in Fig. 2 shows that convergence is obtained for \nc < 0.5 as expected from theoretical analysis. For the MRT model, simulation \nresults given in Fig. 3 shows that convergence is obtained for c < 0.7, also as \nexpected from theoretical analysis. The theoretical location at convergence for \nthe Sutton and Barto model is also shown in Figure 2. It is readily seen that \nthe simulation results confirm the theoretical expectations. \n\n,v ....... \u00b7 \u00b7 .. \u00b7 \u00b7 \u00b7 \u00b7 .. \u00b7 .. \u00b7 .. \u00b7 \u00b7 \u00b7 .. \ni /r \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \u00b7 \n,. '..,...._....-_---------\n\n1: c \u2022 0.1 \n2: c \u2022 0.2 \n3: c \u2022 0.3 \n4: c \u2022 0.4 \ns: c \u00b70.5 \n6: c \u2022 0.6 \n1: c \u2022 0.7 \n\n\"'r.al \nOutput \n\n, \n\nI .\u2022 \n\nI.' \n\n... \n\n.. \u00b7~----~--~M----~JI-----.~--~a~--~. \n\nFigure 2. 'lou or MuroD&l _tpuu YeT.US Ule .... er of 1urat1011& for the \nSuttoa-Barto ~el witb '1frerent .alues of ~aptat1on CODstant c. \n\n\f171 \n\n~ \"\"1\"\" \n\n..... . . . . . . . . . . . . . . . . . . . . . . . . . . . \n................................. \n\n\u2022 \u2022 . \u2022 \u2022 I \u2022 \u2022 , \n\nI 1 \n\nI \n\n... \n\n1.1 \n\nI.' \n\n... \n\nlleuroul \nOutput \n\n., \n\n1: e - 0.1 \n2: e - 0.2 \n3: e - 0.3 \n4: e - 0.4 \nS: e - 0.5 \n6: e - 0.6 \n\n... I~, ____ ~ ____ ~, ____ ~ ____ ~1~\u00b7~,~-~o~,7~ __ ~, \n\u2022 \n\n. \" . a \n\n\u2022 \n\nJu.ber of iteratiOGa \n\nFigure 3. Plotl of oeuroaal outputl .craus the uuaber of iteratious \n\nfor the MaT ~el with different .alues of adantatlon \nI:DDStaut c. \n\nTo illustrate the rate of convergence, we will plot the trajectory of the \ndeviation in synaptic weights from the optimal values in the logarithmic scale \nsince this error is logarithmic as found earlier. The slope of the line yields the \nrate of convergence. The trajectory for the Sutton-Barto Model is given in \nFigure 4 while that for the MRT model is given in Figure 5. It is clear from \nFigure 4 that the trajectory in the logarithmic form is a straight line. The \nslope Rn(A(S-B)) can readily be calculated. The curve for the MRT model \ngiven in Figure 5 is also a straight line but with a much larger slope showing \nfaster convergence. \n\nSUMMARY \n\nIn this paper, we have sought to discover analytically the convergence \nbehavior of three adaptive neuronal models. From the analysis, we see that \nthe Hebb model does not converge at all. With constant active inputs, the \noutput will grow exponentially. In spite of this lack of convergence the Hebb \nmodel is still a workable model realizing that the divergent behavior would \nbe curtailed by the sigmoidal transformation to yield realistic outputs. The \n\n\f172 \n\n'II \n\n,._) \n\nt \n\n.... \nI \n\"uroul \nOutput \nDniatiotl 1 \nI \n\nLto \n\nI \n\n.. \n\n1: \n2: \n3: \n4: \n\n.~ \n\n\\' \\\\ \" \n\\~ '---\n\\ \n\\ \n\\ \n\\ \n\n\\ \n\n'\\ \n\\ \n\n\\ \n\n\\ \n\n\\ \n\n',,--\n\" - \u2022 \n\nII \n\n.u.ber of iterationa \n\ne -.0.1 \nC - 0.2 \ne \u2022 0.3 \ne - 0.4 \n\n'\\ \n\\ \n\n\" \n\n.. \n\n.. \n\n\u2022 \n\nFigure 4. Trajectories of Deuronal output deviationa froa atatic .alues \n\nfor the Sutton-\"rt~ ~el with ~lfferent value. ~f adaptation \ncOIIstallt C. \n\nI.-\n\n80 .. \n\nlleuroD&l. \nOutput \nDeviation \n\nLtl \n\n... \n\n.\"~ \n\n1: C \u00b7 0.1 \n2: C \u00b7 0.2 \n3: c\u00b7 0.3 \n4: C. 0.4 \n\n\\ \n\n~ \\.\\ (\\ \\ \n\\\\ \\ .' , \n\"' \\ \n\\\\ \\ \n' I \\ \\\\ \\ \n! ~ \\ \n\\ \" \n: \" \n, \n\\ \n\\ \n, \n\\ \n\\ \n\" \nNuaber of iterations \n\n, \n\\ \n\\ \n\\ '\\ \n'. \ni \n\\ ~ \n.. \n\\ \ni \n) \n\n.. \n\n, \nn \n\n~ \n\n, .. \n\n'\" \n\nFigure 5. Trajectories of neuronal output deviations fra. atatic \n\nvalues for tbe KRT ~el witb different values of \nadaptation constant c. \n\n\f173 \n\nanalysis on the Sutton and Barto model shows that this model will converge \nwhen the adaptation constant c is carefully chosen. The bounds for c is also \nfound for this model. Due to the structure of this model, both the location at \nconvergence and the rate of convergence are also found. We have also introduced \na new model of neuronal plasticity called the most recent trace (MRT) model. \nCertain similarities exist between the MRT model and the Sutton-Barto model \nand also between the MRT model and the Klopf model. Analysis shows that the \nupdate equations for the synaptic weights are quadratic resulting in polynomial \nrate of convergence. Simulation results also show that much faster convergence \nrate can be obtained with the MRT model. \n\nREFERENCES \n\n1. Sutton, R.S. and A.G. Barto, Psychological Review, vol. 88, p. 135, (1981). \n2. Hageman, L. A. and D.M. Young. Applied Interactive Methods. (Aca(cid:173)\n\ndemic Press, Inc. 1981). \n\n3. Omidvar, Massoud. Analysis of Neuronal Plasticity. Doctoral disserta(cid:173)\n\ntion, School of Electrical Engineering and Computer Science, University of \nOklahoma, 1987. \n\n4. Klopf, A.H. Proceedings of the American Institute of Physics Conference \n\n#151 on Neural Networks for Computing, p. 265-270, (1986). \n\n\f", "award": [], "sourceid": 55, "authors": [{"given_name": "John", "family_name": "Cheung", "institution": null}, {"given_name": "Massoud", "family_name": "Omidvar", "institution": null}]}