{"title": "Optimal Movement Primitives", "book": "Advances in Neural Information Processing Systems", "page_first": 1023, "page_last": 1030, "abstract": null, "full_text": "Optimal Movement Primitives \n\nTerence D. Sanger \n\nJet Propulsion Laboratory \n\n(818) 354-9127 tds@ai.mit.edu \n\nMS 303-310 \n\n4800 Oak Grove Drive \nPasadena, CA 91109 \n\nAbstract \n\nThe theory of Optimal Unsupervised Motor Learning shows how \na network can discover a reduced-order controller for an unknown \nnonlinear system by representing only the most significant modes. \nHere, I extend the theory to apply to command sequences, so that \nthe most significant components discovered by the network corre(cid:173)\nspond to motion \"primitives\". Combinations of these primitives \ncan be used to produce a wide variety of different movements. \nI demonstrate applications to human handwriting decomposition \nand synthesis, as well as to the analysis of electrophysiological \nexperiments on movements resulting from stimulation of the frog \nspinal cord. \n\n1 \n\nINTRODUCTION \n\nThere is much debate within the neuroscience community concerning the inter(cid:173)\nnal representation of movement, and current neurophysiological investigations are \naimed at uncovering these representations. \nIn this paper, I propose a different \napproach that attempts to define the optimal internal representation in terms of \n\"movement primitives\" , and I compare this representation with the observed behav(cid:173)\nior. In this way, we can make strong predictions about internal signal processing. \nDeviations from the predictions can indicate biological constraints or alternative \ngoals that cause the biological system to be suboptimal. \n\nThe concept of a motion primitive is not as well defined as that of a sensory primitive \n\n\f1024 \n\nTerence Sanger \n\nu \n\ny \n\np \n\nz \n\nFigure 1: Unsupervised Motor Learning: The plant P takes inputs u and produces \noutputs y. The sensory map C produces intermediate variables z, which are mapped \nonto the correct command inputs by the motor network N. \n\nwithin the visual system, for example. There is no direct equivalent to the \"receptive \nfield\" concept that has allowed interpretation of sensory recordings. In this paper, I \nwill propose an internal model that involves both motor receptive fields and a set of \nmovement primitives which are combined using a weighted sum to produce a large \nclass of movements. In this way, a small number of well-designed primitives can \ngenerate the full range of desired behaviors. \n\nI have previously developed the concept of \"optimal unsupervised motor learning\" \nto investigate optimal internal representations for instantaneous motor commands. \nThe optimal representations adaptively discover a reduced-order linearizing con(cid:173)\ntroller for an unknown nonlinear plant. The theorems give the optimal solution in \ngeneral, and can be applied to special cases for which both linear and nonlinear \nadaptive algorithms exist (Sanger 1994b). In order to apply the theory to com(cid:173)\nplete movements it needs to be extended slightly, since in general movements exist \nwithin an infinite-dimensional task space rather than a finite-dimensional control \nspace. The goal is to derive a small number of primitives that optimally encode \nthe full set of observed movements. Generation of the internal movement primi(cid:173)\ntives then becomes a data-compression problem, and I will choose primitives that \nminimize the resultant mean-squared error. \n\n2 OPTIMAL UNSUPERVISED MOTOR LEARNING \n\nOptimal Unsupervised Motor Learning is based on three principles: \n\n1. Dimensionality Reduction \n2. Accurate Reduced-order Control \n3. Minimum Sensory error \n\nConsider the system shown in figure 1. At time t, the plant P takes motor inputs u \nand produces sensory outputs y. A sensory mapping C transforms the raw sensory \ndata y to an intermediate representation z. A motor mapping takes desired values \nof z and computes the appropriate command u such that CPu = z. Note that the \n\n\fOptimal Movement Primitives \n\n1025 \n\nloop in the figure is not a feedback-control loop, but is intended to indicate the flow \nof information. With this diagram in mind, we can write the three principles as: \n\n1. dim[z] < dim[y] \n2. GPNz=z \n3. IIPNGy - yll is minimized \n\nWe can prove the following theorems (Sanger 1994b): \nTheorem 1: For all G there exists an N such that G P N z = z. If G is linear and \np- 1 is linear, then N is linear. \n\nTheorem 2: For any G, define an invertible map C such that GC-l =_1 on range[G]. \nThen liP NGy - yll is minimized when G is chosen such that Ily - G-1GII is mini(cid:173)\nmized. If G and P are linear and the singular value decomposition of P is given by \nLT SR, then the optimal maps are G = Land N = RT S-I. \nFor the discussion of movement, the linear case will be the most important since in \nthe nonlinear case we can use unsupervised motor learning to perform dimensional(cid:173)\nity reduction and linearization of the plant at each time t. The movement problem \nthen becomes an infinite-dimensional linear problem. \n\nPreviously, I have developed two iterative algorithms for computing the singular \nvalue decomposition from input/output samples (Sanger 1994a). The algorithms are \ncalled the \"Double Generalized Hebbian Algorithm\" (DGHA) and the \"Orthogonal \nAsymmetric Encoder\" (OAE). DGHA is given by \n\n8G \n8NT \n\n'Y(zyT - LT[zzT]G) \n'Y(zuT - LT[zzT]NT ) \n\nwhile OAE is described by: \n\n'Y(iyT - LT[ZiT]G) \n'Y(Gy - LT[GGT]z)uT \n\nwhere LT[ ] is an operator that sets the above diagonal elements of its matrix \nargument to zero, y = Pu, z = Gy, z = NT u, and 'Y is a learning rate constant. \nBoth algorithms cause G to converge to the matrix of left singular vectors of P, and \nN to converge to the matrix of right singular vectors of P (multiplied by a diagonal \nmatrix for DGHA) . DGHA is used in the examples below. \n\n3 MOVEMENT \n\nIn order to extend the above discussion to allow adaptive discovery of movement \nprimitives, we now consider the plant P to be a mapping from command sequences \nu(t) to sensory sequences y(t). We will assume that the plant has been feedback \nlinearized (perhaps by unsupervised motor learning). We also assume that the \nsensory network G is constrained to be linear. In this case, the optimal motor \nnetwork N will also be linear. The intermediate variables z will be represented by \na vector. The sensory mapping consists of a set of sensory \"receptive fields\" gi(t) \n\n\f1026 \n\nTerence Sanger \n\nA .. \n\nMotor Map \n\nSensory Map \n\nI n1(t)zl +- Zl +-\n\n---lEt): \n\n: \n\n\\ nn(~)~ ~ ~n +-\n\nFigure 2: Extension of unsupervised motor learning to the case of trajectories. \nPlant input and output are time-sequences u(t) and y(t). The sensory and motor \nmaps now consist of sensory primitives gi(t) and motor primitives ni(t). \n\nsuch that \n\nZi = J gj(t)y(t)dt =< gily > \n\nand the motor mapping consists of a set of \"motor primitives\" ni(t) such that \n\nu(t) = L ni(t)zi \n\ni \n\nas in figure 2. If the plant is equal to the identity (complete feedback linearization), \nthen gi(t) = ni(t). In this case, the optimal sensory-motor primitives are given by \nthe eigenfunctions of the autocorrelation function of y(t) . If the autocorrelation is \nstationary, then the infinite-window eigenfunctions will be sinusoids. Note that the \noptimal primitives depend both on the plant P as well as the statistical distribution \nof outputs y(t). \nIn practice, both u(t) and y(t) are sampled at discrete time-points {tic} over a finite \ntime-window, so that the plant input and output is in actuality a long vector. Since \nthe plant is linear, the optimal solution is given by the singular value decomposition, \nand either the DGHA or OAE algorithms can be used directly. The resulting sensory \nprimitives map the sensory information y(t) onto the finite-dimensional z, which \nis usually a significant data compression. The motor primitives map Z onto the \nsequence u(t), and the resulting y(t) = P[u(t)] will be a linear projection of y(t) \nonto the space spanned by the set {Pni(t)}. \n\n4 EXAMPLE 1: HANDWRITING \n\nAs a simple illustration, I examine the case of human handwriting. We can consider \nthe plant to be the identity mapping from pen position to pen position, and the \n\n\fOptimal Movement Primitives \n\n1027 \n\n1. \n\n2. \n\n3. \n\n4. \n\n5. \n\n6. \n\n7. \n\n8. \n\nFigure 3: Movement primitives for sampled human handwriting. \n\n\f1028 \n\nTerence Sanger \n\nhuman to be taking desired sensory values of pen position and converting them \ninto motor commands to move the pen. The sensory statistics then reflect the \nset of trajectories used in producing handwritten letters. An optimal reduced-order \ncontrol system can be designed based on the observed statistics, and its performance \ncan be compared to human performance. \n\nFor this example, I chose sampled data from 87 different examples of lower-case \nletters written by a single person, and represented as horizontal and vertical pen \nposition at each point in time. Blocks of 128 sequential points were used for training, \nand 8 internal variables Zi were used for each of the two components of pen position. \nThe training set consisted of 5000 randomly chosen samples. Since the plant is the \nidentity, the sensory and motor primitives are the same, and these are shown as \n\"strokes\" in figure 3. Linear combinations of these strokes can be used to generate \npen paths for drawing lowercase letters. This is shown in figure 4, where the word \n\"hello\" (not present in the training set) is written and projected using increasing \nnumbers of intermediate variables Zi. The bottom of figure 4 shows the sequence of \nvalues of Zi that was used (horizontal component only). \n\nGood reproduction of the test word was achieved with 5 movement primitives. \nA total of 7 128-point segments was projected , and these were recombined using \nsmooth 50% overlap . Each segment was encoded by 5 coefficients for each of the \nhorizontal and vertical components, giving a total of 70 coefficients to represent \n1792 data points (896 horizontal and vertical components) , for a compression ratio \nof 25 :1. \n\n5 EXAMPLE 2: FROG SPINAL CORD \n\nThe second example models some interesting and unexplained neurophysiological \nresults from microstimulation of the frog spinal cord. (Bizzi et al. 1991) measured \nthe pattern of forces produced by the frog hindlimb at various positions in the \nworkspace during stimulation of spinal interneurons. The resulting force-fields often \nhave a stable\" equilibrium point\" , and in some cases this equilibrium point follows \na smooth closed trajectory during tonic stimulation of the interneuron. However, \nonly a small number of different force field shapes have been found, and an even \nsmaller number of different trajectory types. A hypothesis to explain this result \nis that larger classes of different trajectories can be formed by combining the pat(cid:173)\nterns produced by these cells. This hypothesis can be modelled using the optimal \nmovement primitives described above. \n\nFigure 5a shows a simulation of the frog leg. To train the network, random smooth \nplanar movements were made for 5000 time points. The plant output was considered \nto be 32 successive cartesian endpoint positions, and the plant input was the time(cid:173)\nvarying force vector field. Two hidden units Z were used. In figure 5b we see an \nexample of the two equilibrium point trajectories (movement primitives) that were \nlearned by DG HA. Linear combinations of these trajectories account for over 96% of \nthe variance of the training data, and they can approximate a large class of smooth \nmovements . Note that many other pairs of orthogonal trajectories can accomplish \nthis, and different trials often produced different orthogonal trajectory shapes. \n\n\fOptimal Movement Primitives \n\n1029 \n\nOriginal \n\n1. \n\n2 \n\n3. \n\nCoefficients \n\n5.~ \n\n8.~~ \n\nFigure 4: Projection of test-word \"hello\" using increasing numbers of intermediate \nvariables Zi. \n\n\f1030 \n\no. \n\nWorkSpace \n\nb. \n\nTerence Sanger \n\nFigure 5: a. Simulation of frog leg configuration. b. An example of learned optimal \nmovement primitives. \n\n6 CONCLUSION \n\nThe examples are not meant to provide detailed models of internal processing for \nhuman or frog motor control. Rather, they are intended to illustrate the concept \nof optimal primitives and perhaps guide the search for neurophysiological and psy(cid:173)\nchophysical correlates of these primitives. The first example shows that generation \nof the lower-case alphabet can be accomplished with approximately 10 coefficients \nper letter, and that this covers a considerable range of variability in character pro(cid:173)\nduction. The second example demonstrates that an adaptive algorithm allows the \npossibility for the frog spinal cord to control movement using a very small number \nof internal variables. \n\nOptimal unsupervised motor learning thus provides a descriptive model for the \ngeneration of a large class of movements using a compressed internal description. A \nset of fixed movement primitives can be combined linearly to produce the necessary \nmotor commands, and the optimal choice of these primitives assures that the error \nin the resulting movement will be minimized. \n\nReferences \n\nBizzi E., Mussa-Ivaldi F. A., Giszter S., 1991, Computations underlying the execu(cid:173)\ntion of movement: A biological perspective, Science, 253:287-29l. \nSanger T. D., 1994a, Two algorithms for iterative computation of the singular \nvalue decomposition from input/output samples, In Touretzky D., ed., Advances in \nNeural Information Processing 6, Morgan Kaufmann, San Mateo, CA, in press. \nSanger T. D., 1994b, Optimal unsupervised motor learning, IEEE Trans. Neural \nNetworks, in press. \n\n\f", "award": [], "sourceid": 927, "authors": [{"given_name": "Terence", "family_name": "Sanger", "institution": null}]}