{"title": "Design and Implementation of a High Speed CMAC Neural Network Using Programmable CMOS Logic Cell Arrays", "book": "Advances in Neural Information Processing Systems", "page_first": 1022, "page_last": 1027, "abstract": null, "full_text": "Design and Implementation of a High Speed \nCMAC Neural Network Using Programmable \n\nCMOS Logic Cell Arrays \n\nW. Thomas Miller, III, Brian A. Box, and Erich C. Whitney \n\nDepartment of Electrical and Computer Engineering \n\nKingsbury Hall \n\nUniversity of New Hampshire \nDurham, New Hampshire 03824 \n\nJames M. Glynn \n\nShenandoah Systems Company \n\n1A Newington Park \n\nWest Park Drive \n\nNewington, New Hampshire 03801 \n\nAbstract \n\nA high speed implementation of the CMAC neural network was designed \nusing dedicated CMOS logic. This technology was then used to implement \ntwo general purpose CMAC associative memory boards for the VME bus. \nEach board implements up to 8 independent CMAC networks with a total \nof one million adjustable weights. Each CMAC network can be configured \nto have from 1 to 512 integer inputs and from 1 to 8 integer outputs. \nResponse times for typical CMAC networks are well below 1 millisecond, \nmaking the networks sufficiently fast for most robot control problems, and \nmany pattern recognition and signal processing problems. \n\n1 \n\nINTRODUCTION \n\nWe have been investigating learning techniques for the control of robotic manipu(cid:173)\nlators which utilize extensions of the CMAC neural network as developed by Albus \n\n1022 \n\n\fDesign and Implementation of a High Speed CMAC Neural Network \n\n1023 \n\n(1972; 1975; 1979). The learning control techniques proposed have been studied \nin our laboratory in a series of real time experimental studies (Miller, 1986; 1987; \n1989; Miller et al., 1987; 1988; 1990). These studies successfully demonstrated the \nability to learn the kinematics of a robot/video camera system interacting with \nrandomly oriented objects on a moving conveyor, and to learn the dynamics of a \nmulti-axis industrial robot during high speed motions. We have also investigated \nthe use of CMAC networks for pattern recognition (Glanz and Miller, 1987; Herold \net al., 1988) and signal processing (Glanz and Miller, 1989) applications, with en(cid:173)\ncouraging results. The primary goal of this project was to implement a compact, \nhigh speed version of the CMAC neural network using CMOS logic cell arrays. Two \nprototype CMAC associative memory systems for the industry standard VME bus \nwere then constructed. \n\n2 THE CMAC NEURAL NETWORK \n\nFigure 1 shows a simple example of a CMAC network with two inputs and one \noutput. Each variable in the input state vector is fed to a series of input sensors \nwith overlapping receptive fields. The width of the receptive field of each sensor \nproduces input generalization, while the offset of the adjacent fields produces input \nquantization. The binary outputs of the input sensors are combined in a series of \nthreshold logic units (called state space detectors) with thresholds adjusted to pro(cid:173)\nduce logical AND functions. Each of these units receives one input from the group \nof sensors for each input variable, and thus its input receptive field is the interior \nof a hypercube in the input hyperspace. The input sensors are interconnected in \na sparse and regular fashion, so that each input vector excites a fixed number of \nstate space detectors. The outputs of the state space detectors are connected ran(cid:173)\ndomly to a smaller set of threshold logic units (called multiple field detectors) with \nthresholds adjusted such that the output will be on if any input is on. The receptive \nfield of each of these units is thus the union of the fields of many of the state space \ndetectors. Finally, the output of each multiple field detector is connected, through \nan adjustable weight, to an output summing unit. The output for a given input is \nthus the sum of the weights selected by the excited multiple field detectors. \n\nThe nonlinear nature of the CMAC network is embodied in the interconnections of \nthe input sensors, state space detectors, and multiple field detectors, which perform \na fixed nonlinear associative mapping of the continuous valued input vector to a \nmany dimensional binary valued vector (which has tens or hundreds of thousands \nof dimensions in typical implementations). The adaptation problem is linear in this \nmany dimensional space, and all of the convergence theorems for linear adaptive \nelements apply. \n\n3 THE CMAC HARDWARE DESIGN \n\nThe custom implementation of the CMAC associative memory required the devel(cid:173)\nopment of two devices. The first device performs the input associative mapping, \nconverting application relevant input vectors into traditional RAM addresses. The \nsecond device performs CMAC response accumulation, summing the weights from \nall excited receptive fields. Both devices were implemented using 70 MHz XILINX \n\n\f1024 Miller, Box, Whitney, and Glynn \n\nInput \nSensors \n\n\u2022 . E \n\nState Space \nDetectors \n\nt \n\nWeights \n\n'S \nQ. \nC \n\nC'oI \n'S \nQ. \nC \n\nc \nIi \n\nf(\u00a7) \n\nC> \n\nMultiple Field \nDetectors \n\n110 Total \n\nUnits \n\no Logical AND unit \nc> Logical OR unit \n\nFigure 1: A Simple Example of a CMAC Neural Network \n\n3090 programmable logic cell arrays. \n\nThe associative mapping device uses a bit recursive mapping scheme developed at \nUNH, which is similar in philosophy to the CMAC mapping proposed by Albus, \nbut is structured for efficient implementation using discrete logic. The\" address\" of \neach excited virtual receptive field is formed recursively by clocking the input vector \ncomponents sequentially from a buffer FIFO. The hashing of the virtual receptive \nfield address to a physical RAM address is performed simultaneously, using pipelined \nlogic. The resulting associative mapping generates one 18 bit RAM address for a \ngiven input vector. The multiple addresses, corresponding to the multiple receptive \nfields excited by a single input vector could be generated simultaneously using \nparallel addressing circuits, or sequentially using a single circuit. \n\nThe second CMAC device serves basically as an accumulator during CMAC response \ngeneration. As successive addresses are produced by the associative mapping circuit, \nthe accumulator sums the corresponding values from the data RAM. During memory \ntraining, the response accumulation circuit adds the training adjustment to each \nof the addressed memory locations, placing the result back in the RAM. Eight \nindependent CMAC output channels were placed on a single device. \n\nIn the final VME system design (Figure 2), a single CMAC associative mapping \ndevice was used. Overlapping receptive fields were implemented sequentially using \nthe same device. A single CMAC response accumulation device was used, providing \neight parallel output channels. A weight vector memory containing 1 million 8 \nbit weights was provided using 85 nanosecond 512 KByte static RAM SIMMs. \nA TMS320E15 micro controller was utilized to supervise communications with the \nVME bus. The operational firmware for the micro controller chip was designed to \n\n\fDesign and Implementation of a High Speed CMAC Neural Network \n\n1025 \n\nCMAC Associative Mapping \n\nCMAC Output Accumulators \n\nVME PI Connector \n\nFigure 2: The Component Side of the VME Based CMAC Associative 1\\lemory \nCard. The two large XILINX 3090 logic cell arrays implement the CMAC associative \nmapping and the response accumulation/weight adjustment circuitry. The weights \nare stored in the 1 Mbyte static RAM. The TMS320E15 microcontroller supervises \ncommunications between the CMAC hardware and the VME host. \n\nprovide maximum flexibility in the logical organization of the CMAC associative \nmemory, as viewed by the VME host system. The board can be initialized to act as \nfrom 1 to 8 independent virtual CMAC networks. For each network, the number of \n16 bit inputs is selectable from 1 to 512, the number of 16 bit outputs is selectable \nfrom 1 to 8, and the number of overlapping receptive fields is selectable from 2 to \n256. \n\nFigure 3 shows typical response times during training and response generation op(cid:173)\nerations for a CMAC network with 1 million adjustable weights. The data shown \nrepresent networks with 32 integer inputs and 8 integer outputs, with the num(cid:173)\nber of overlapping receptive fields varied between 8 and 256. Throughout most of \nthis range CMAC training and response times are well below 1 millisecond. These \nperformance specifications should accommodate typical real time control problems \n(allowing 1000 cycle per second control rates), as well as many problems in pattern \nrecognition. \n\nA similar CMAC system for the 16 bit PC-AT bus has been developed by the \nShenandoah Systems Company for commercial applications. This CMAC system \nsupports both 8 and 16 bit adjustable weights (1 Mbyte total storage), and 8 inde(cid:173)\npendent virtual CMAC networks on a single card. Response times for the commer(cid:173)\ncial CMAC-AT card are similar to those shown in Figure 3. A commercial version \n\n\f1026 Miller, Box, Whitney, and Glynn \n\n\u00b7\u00b7\u00b7T\u00b7\u00b7T\u00b7rTT\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7c\u00b7i1~c\u00b7\u00b7\u00b7\u00b7\u00b7~\u00b7fM\u00b7~n\u00b7~\u00b7\u00b7\u00b7\u00b7p\u00b7~\u00b7~\u00b7~\u00b7T\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7r\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7j \n\u00b7\u00b7\u00b7\u00b7+\u00b7\u00b7\u00b7\u00b7j\u00b7\u00b7\u00b7\u00b7i\u00b7\u00b7\u00b7j\u00b7\u00b7\u00b7j\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7 32 Inputs \u2022 a (kitputs' \u2022 1'ltiilian '!Ie'ights ................. + ............. j \nI \n\n! II ! I 1 ~lliJc.nd I I IIIII1 \n\nI \n\nI -... \nI \n\n11+81 \n\n11+82 \n\nFigure 3: CMAC Associative Memory Response and Training Times. Response \ntimes are shown for values of the generalization parameter (the number of overlap(cid:173)\nping receptive fields) between 8 and 256. In each case the CMAC had 32 integel' \ninputs, 8 integer outputs, and one million adjustable weights. \n\nof the VME bus design is currently under development . \n\nAcknow ledgements \n\nThis work was sponsored in part by the Office of Naval Research (ONR Grant \nNumber N00014-89-J-1686) and the National Institute of Standards and Technology. \n\nReferences \n\nAlbus, J. S., Theoretical and Experimental Aspects of a Cerebellar Model. PhD \nThesis, University of Maryland, Dec . 1972. \n\nAlbus, J. S., A New Approach to Manipulator Control: The Cerebellar Model Ar(cid:173)\nticulation Controller (CMAC). Trans. of the ASME, Journal of Dynamic Systems, \nMeasurement and Control, vol. 97, pp. 220-227, September, 1975. \n\nAlbus, J. S., Mechanisms of Planning and Problem Solving in the Brain. Mathe(cid:173)\nmatical Biosciences, vol. 45, pp. 247-293, August, 1979. \n\nMiller, W. T., A Nonlinear Learning Controller for Robotic Manipulators. Proc. of \nthe SPIE: Intelligent Robots and Computer Vision, vol 726, pp . 416-423, October, \n1986. \n\n\fDesign and Implementation of a High Speed CMAC Neural Network \n\n1027 \n\nMiller, W. T., Sensor Based Control of Robotic Manipulators Using A General \nLearning Algorithm. IEEE J. of Robotics and Automation, vol. RA-3, pp. 157-\n165, April, 1987. \nMiller, W. T., Glanz, F. H., and Kraft, 1. G., Application of a General Learning \nAlgorithm to the Control of Robotic Manipulators. The International Journal of \nRobotics Research, vol. 6.2, pp. 84-98, Summer, 1987. \nMiller, W.T., and Hewes, R.P., Real Time Experiments in Neural Network Based \nLearning Control During High Speed, Nonrepetitive Robot Operations. Proceedings \nof the Third IEEE International Symposium on Intelligent Control, Washington, \nD.C., August 24-26, 1988. \n\nMiller, W, T., Real Time Application of Neural Networks for Sensor-Based Control \nof Robots with Vision. IEEE Transactions on Systems, Man, and Cybernetics. \nSpecial issue on Information Technology for Sensory-Based Robot Manipulators, \nvol. 19, pp. 825-831, 1989. \nMiller, W. T., Hewes, R. P., Glanz, F. H., and Kraft, 1. G., Real Time Dynamic \nControl of an Industrial Manipulator Using a Neural Network Based Learning Con(cid:173)\ntroller. IEEE J. of Robotics and Automation vol. 6, pp. 1-9, 1990. \nGlanz, F. H., Miller, W. T., Shape Recognition Using a CMAC Based Learning \nSystem. Proceedings SPIE: Intelligent Robots and Computer Vision, Cambridge, \nMass., Nov., 1987. \nHerold, D. J., Miller, W. T., Kraft, L. G., and Glanz, F. H., Pattern Recognition \nUsing a CMAC Based Learning System. Proceedings SPIE: Automated Inspection \nand High Speed Vision Architectures II, vol. 1004, pp. 84-90, 1988. \nGlanz, F. H., and Miller, W. T., Deconvolution and Nonlinear Inverse Filtering \nUsing a Neural Network. Proc. ICASSP 89, Glasgow, Scotland, May 23-26, 1989, \nvol. 4, pp. 2349-2352. \n\n\f", "award": [], "sourceid": 404, "authors": [{"given_name": "W.", "family_name": "Miller", "institution": null}, {"given_name": "Brian", "family_name": "Box", "institution": null}, {"given_name": "Erich", "family_name": "Whitney", "institution": null}, {"given_name": "James", "family_name": "Glynn", "institution": null}]}