{"title": "WATTLE: A Trainable Gain Analogue VLSI Neural Network", "book": "Advances in Neural Information Processing Systems", "page_first": 874, "page_last": 881, "abstract": null, "full_text": "WATTLE: A Trainable Gain Analogue \n\nVLSI Neural Network \n\nRichard Coggins and Marwan Jabri \n\nSystems Engineering and Design Automation Laboratory \n\nDepartment of Electrical Engineering J03, \n\nUniversity of Sydney, 2006. \n\nAustralia. \n\nEmail: richardc@sedal.su.oz.au \n\nmarwan@sedal.su.oz.au \n\nAbstract \n\nThis paper describes a low power analogue VLSI neural network \ncalled Wattle. Wattle is a 10:6:4 three layer perceptron with multi(cid:173)\nplying DAC synapses and on chip switched capacitor neurons fabri(cid:173)\ncated in 1.2um CMOS. The on chip neurons facillitate variable gain \nper neuron and lower energy/connection than for previous designs. \nThe intended application of this chip is Intra Cardiac Electrogram \nclassification as part of an implantable pacemaker / defibrillator sys(cid:173)\ntem. Measurements of t.he chip indicate that 10pJ per connection \nis achievable as part of an integrated system. Wattle has been suc(cid:173)\ncessfully trained in loop on parity 4 and ICEG morphology classi(cid:173)\nfication problems. \n\n1 \n\nINTRODUCTION \n\nA three layer analogue VLSI perceptron has been previously developed by \n[Leong and Jabri, 1993]. This chip named Kakadu uses 6 bit digital weight storage, \nmultiplying DACs in the synapses and fixed value off chip resistive neurons. The \nchip described in this paper called Wattle has the same synapse arrays as Kakadu, \nhowever, has the neurons implemented as switched capacitors on chip. For both \nKakadu and Wattle, analogue techniques have been favoured as they offer greater \nopportunity to achieve a low energy and small area design over standard digital \n\n874 \n\n\fWATfLE: A Trainable Gain Analogue VLSI Neural Network \n\n875 \n\n----------------------\n, \n, , \n\n: NEURON CIRCUIT \n\nlout+ ODd Jout- 0UlpUt \n\n&_-.,..- -=~-~ \n\nc:oonoct ..... \n\n------------------------------------ -_.-- --------------------, \n, , , , \n\nI \n\n~ \n\nWEIGHT STORAGE \n\n--------------------- -1 \n\nHII \n\nmAC \n\nSYNAPSE CIRCUIT \n\n00 \n\nII \n\nI \n\n, ~----------------------\n, \n: \n\nI \nI \nI L ______________________________________________________________ J \n\nFigure 1: Wattle Synapse Circuit Diagram \n\ntechniques since the transistor count for the synapse can be much lower and the \ncircuits may be biased in subthreshold. Some work has been done in the low en(cid:173)\nergy digital area using subthreshold and optimised threshold techniques, however \nno large scale circuits have been reported so far. [Burr and Peterson, 1991] The cost \nof using analogue techniques is however, increased design complexity, sensitivity to \nnoise, offsets and component tolerances. In this paper we demonstrate that difficult \nnonlinear problems and real world problems can be trained despite these effects. \n\nAt present, commercially available pacemakers and defibrillators use timing deci(cid:173)\nsion trees implemented on CMOS microprocessors for cardiac arrythmia detection \nvia peak detection on a single ventricular lead. Even when atrial leads are used, In(cid:173)\ntra Cardiac Electrogram (ICEG) morphology classification is required to separate \nsome potentially fatal rhythms from harmless ones. [Leong and J abri, 1992] The \nrequirements of such a morphology classifier are: \n\n\u2022 Adaptable to differing morphology within and across patients. \n\n\u2022 Very low power consumption. ie. minimum energy used per classification. \n\n\u2022 Small size and high reliability. \n\nThis paper demonstrates how this morphology classification may be done using a \nneural network architecture and thereby meet the constraints of the implantable \narrythmia classification system. In addition, in loop training results will also be \ngiven for parity 4, another difficult nonlinear training problem. \n\n\f876 \n\nCoggins and Jabri \n\nVdd \n\nreset \nclock \n\ns \n\ns \n\n~ ~_c_lk_0 __ -+ __________ ~ ______ ~ __________ ~ \n\nt------------------t---------'co='-p -lL----> fan outto \n; next layer \n\nCOM \n\ncharging \nclock \n\n'\" \n\nsynapse \nrow \nconnects , 1i::)>----_CIU _____________ _______ -----' \n\nCIP \n\nFigure 2: Wattle Neuron Circuit Diagram \n\n-\n\nRow Addrus \n\n10x6 Synapse Array \n\nI hi muHlplexor I \n\nneuron. \nIndkclemux I \nDD \nDO \nDO \nDO \nneurons D D \n.----.Dar\"'O'ta---=Re,.-g-.I,....., .. :-e-r -\"\"1 Buffers \n\n6x4Synapse \n\nArray \n\n'---\nColumn \nAddrus \n~ \n~ \n\nFigure 3: Wattle Floor Plan \n\n\fWATTLE: A Trainable Gain Analogue VLSI Neural Network \n\n877 \n\n.. \n\n611\"',. \n\nFigure 4: Photomicrograph of Wattle \n\n2 ARCHITECTURE \n\nSwitched capacitors were chosen for the neurons on Wattle after a test chip was fab(cid:173)\nricated to evaluate three neuron designs. [Coggins and Jabri, 1993] The switched \ncapacitor design was chosen as it allowed flexible gain control of each neuron, in(cid:173)\nvestigation of gain optimisation during limited precision in loop training and the \nrealisation of very high effective resistances. The wide gain range of the switched \ncapacitor neurons and the fact that they are implemented on chip has allowed Wat(cid:173)\ntle to operate over a very wide range of bias currents from 1 pA LSB DAC current \nto 10nA LSB DAC current. \n\nSignalling on Wattle is fully differential to reduce the effect of common mode noise. \nThe synapse is a multiplying digital to analogue convertor with six bit weights. The \nsynapse is shown in figure L This is identical to the synapse used on the Kakadu \nchip [Leong and Jabri, 1993]. The MDAC synapses use a weighted current source \nto generate the current references for the weights. The neuron circuit is shown in \nfigure 2. The neuron requires reset and charging clocks. The period of the charging \nclock determines the gain. Buffers are used to drive the neuron outputs off chip to \navoid the effects of stray pad capacitances. \n\nFigure 3 shows a floor plan of the wattle chip. The address and data for the weights \naccess is serial and is implemented by the shift registers on the boundary of the chip. \nThe hidden layer multiplexor allows access to the hidden layer neuron outputs. The \nneuron demultiplexor switches the neuron clocks between the hidden and output \nlayers. Figure 4 shows a photomicrograph of the wattle die. \n\n3 ELECTRICAL CHARACTERISTICS \n\nTests have been performed to verify the operation of the weighted current source \nfor the MDAC synapse arrays, the synapses, the neurons and the buffers driving the \nneuron voltages off chip. The influences of noise, offsets, crosstalk and bandwidth \nof these different elements have been measured. In particular, the system level noise \nmeasurement showed that the signal to noise ratio was 40dB. A summary of the \nelectrical characteristics appears in table L \n\n\f878 \n\nCoggins and Jabri \n\nTable 1: Electrical Characteristics and Specifications \n\nParameter \n\nValue \n\nComment \n\n2.2 x 2.2mm~ \n1.2um Nwell CMOS 2M2P \nweights 6bit, gains 7bit \n\nArea \nTechnology \nResolution \nEnergy per connection 43pJ \nLSB DAC current \nFeedforward delay \nSynapse Offset \nGain cross talk delta \n\n200pA \n1.5ms \n5mV \n20% \n\nstandard process \nweights on chip, gains off \nall weights maximum \ntypical \n@200pA, 3V supply \ntypical maximum \nmaxImum \n\nA gain cross talk effect between the neurons was discovered during the electrical \ntesting. The mechanism for this cross talk was found to be transients induced on the \ncurrent source reference lines going to all the synapses as individual neuron gains \ntimed out. The worst case cross talk coupled to a hidden layer neuron was found \nto be a 20% deviation from the singularly activated value. However, the training \nresults of the chip do not appear to suffer significantly from this effect. \nA related effect is the length of time for the precharging of the current summation \nlines feeding each neuron due to the same transients being coupled onto the current \nsource when each neuron is active. The implication of this is an increase in energy \nper classification for the network due to the transient decay time. However, one of \nthe current reference lines was available on an outside pin, so the operation of the \nnetwork free of these transients could also be measured. For this design, including \nthe transient conditions, an energy per connection of 43pJ can be achieved. This \nmay be reduced to 10pJ by modifying the current source to reduce transients and \nneglecting the energy of the buffers. This is to be compared with typical digital \nlOnJ per connection and analogue of 60pJ per connection appearing in the literature. \n[Delcorso et. al., 1993], Table 1. \n\n4 TRAINING BOTH GAINS AND WEIGHTS \n\nA diagram of the system used to train the chip is shown in figure 5. The training \nsoftware is part of a package called MUME [J abri et. al., 1992], which is a multi \nmodule neural network simulation environment. Wattle is interfaced to the work \nstation by Jiggle, a general purpose analogue and digital chip tester developed by \nSEDAL. Wattle, along with gain counter circuitry, is mounted on a separate daugh(cid:173)\nter board which plugs into Jiggle. This provides a software configurable testing \nenvironment for Wattle. In loop training then proceeds via a hardware specific \nmodule in MUME which writes the weights and reads back the analogue output of \nthe chip. Wattle can then be trained by a wide variety of algorithms available in \nMUME. \n\nWattle has been trained in loop using a variation on the Combined Search Algorithm \n(CSA) for limited precision training. [Xie and Jabri, 1992] (Combination of weight \nperturbation and axial random search). The variation consists of training the gains \n\n\f", "award": [], "sourceid": 755, "authors": [{"given_name": "Richard", "family_name": "Coggins", "institution": null}, {"given_name": "Marwan", "family_name": "Jabri", "institution": null}]}