{"title": "Modeling Small Oscillating Biological Networks in Analog VLSI", "book": "Advances in Neural Information Processing Systems", "page_first": 384, "page_last": 393, "abstract": null, "full_text": "384 \n\nMODELING  SMALL  OSCILLATING \n\nBIOLOGICAL NETWORKS IN ANALOG  VLSI \n\nSylvie  Ryckebusch,  James  M.  Bower,  and Carver Mead \n\nCalifornia Instit ute of Technology \n\nPasadena,  CA 91125 \n\nABSTRACT \n\nWe  have used analog VLSI technology to model a class of small os(cid:173)\ncillating  biological neural circuits  known  as  central pattern  gener(cid:173)\nators  (CPG). These circuits generate rhythmic patterns of activity \nwhich drive locomotor behaviour in the animal.  We  have designed, \nfabricated,  and tested a model neuron circuit which relies on many \nof the  same  mechanisms  as  a  biological  central pattern  generator \nneuron,  such  as  delays  and  internal feedback.  We  show  that  this \nneuron can be used  to build several small circuits based on known \nbiological CPG circuits, and that these circuits produce patterns of \noutput which  are very similar to the observed biological patterns. \n\nTo  date,  researchers  in  applied  neural  networks  have  tended  to  focus  on  mam(cid:173)\nmalian systems  as  the  primary source  of  potentially  useful  biological  information. \nHowever,  invertebrate systems may represent  a source  of ideas  in many ways  more \nappropriate, given current levels of engineering sophistication in building neural-like \nsystems, and given the state of biological understanding of mammalian circuits.  In(cid:173)\nvertebrate  systems  are  based  on  orders  of magnitude  smaller  numbers  of neurons \nthan  are  mammalian  systems.  The  networks  we  will  consider  here,  for  example, \nare  composed  of  about  a  dozen  neurons,  which  is  well  within  the  demonstrated \ncapabilities  of current  hardware  fabrication  techniques.  Furthermore,  since  much \nmore detailed structural information is  available about these systems than for  most \nsystems in  higher animals, insights can be guided by real information rather than by \nguesswork.  Finally, even though they are constructed of small numbers of neurons, \nthese networks have  numerous interesting and  potentially even useful properties. \n\nCENTRAL PATTERN  GENERATORS \n\nOf  all  the  invertebrate  neural  networks  currently  being  investigated  by  neurobi(cid:173)\nologists,  the  class  of  networks  known  as  central  pattern  generators  (CPGs)  may \nbe  especially  worthy  of attention.  A  CPG is  responsible  for  generating  oscillatory \nneural  activity  that  governs  specific  patterns  of  motor  output,  and  can  generate \nits  pattern  of activity  when  isolated  from  its  normal neuronal  inputs.  This  prop-\n\n\fModeling Small Oscillating Biological Networks \n\n385 \n\nerty, which greatly facilitates experiments, has enabled biologists to describe several \nCPGs in  detail at  the cellular and synaptic level.  These networks have  been found \nin all animals,  but have been extensively studied in invertebrates [Selverston,  1985]. \n\nWe  chose to model several small CPG networks using analog VLSI technology.  Our \nmodel differs from  most computer simulation models of biological networks [Wilson \nand Bower, in press] in that we did not attempt to model the details of the individual \nionic currents,  nor did we  attempt to model each known connection in the networks. \nRather, our aim  was  to determine the basic  functionality  of a  set of CPG networks \nby modeling them as  the minimum set of connections required to reproduce output \nqualitatively similar to that produced by the real network under certain conditions. \n\nMODELING  CPG NEURONS \n\nThe  basic  building  block  for  our  model  is  a  general  purpose  CPG  neuron  circuit. \nThis circuit, shown  in  Figure  1,  is  our model  for  a  typical neuron  found  in central \npattern  generators,  and  contains  some  of  the  essential  elements  of real  biological \nneurons.  Like real neurons, this model integrates current and uses positive feedback \nto  output  a  train  of pulses,  or  action  potentials,  whose  frequency  depends  on  the \nmagnitude of the current input.  The part of the circuit which generates these pulses \nis  shown  in  Figure 2a [Mead,  19891. \n\nThe second element in  the CPG neuron circuit is the synapse.  In Figure 1,  each pair \nof transistors functions as a synapse.  The p-well transistors are. excitatory synapses, \nwhereas the n-well transistors are inhibitory synapses.  One of the transistors in the \npair sets the strength of the synapse,  while  the other transistor is  the  input of the \nsynapse.  Each  CPG neuron  has four  different synapses. \n\nThe  third element of our model  CPG neuron  involves  temporal delays.  Delays  are \nan essential element in the function of CPGs, and biology has evolved many different \nmechanisms  to  introduce delays  into neural networks.  The membrane capacitance \nof the  cell  body,  different  rates of chemical reactions,  and  axonal transmission  are \njust  a  few  of the  mechanisms which  have  time constants  associated with  them.  In \nour  model we  have  included  synaptic  delay  as  the  principle source of delay  in  the \nnetwork.  This  is  modeled  as  an  RC  delay,  implemented  by  the follower-integrator \ncircuit shown in Figure 2b [Mead,  19891.  The time constant of the delay is a function \nof the  conductance  of  the  amplifier,  set  by  the  bias  G.  A  multiple  time  constant \ndelay line is  formed  by cascading several of these elements.  Our neuron circuit uses \na  delay  line  with  three  time  constants.  The  synapses  which  are  before  the  delay \nelement  are  slow synapses, whereas the  undelayed synapses  are  fa.st  synapses. \n\nWe fabricated the circuit shown in Figure 1 using CMOS, VLSI technology.  Several \nof these  circuits  were  put  on  each  chip,  with  all  of  the  inputs  and  controls  going \nout  to pads, so  that these  cells  could  be externally connected  to form  the  network \nof interest. \n\n\f386 \n\nRyckebusch, Bower, and Mead \n\nslow \n\nexcitation \n\n-t \n\nYout. \n\nG \n\npulse  length \n\nFigure 1.  The CPG neuron  circuit. \n\nr \n\nYout.  -1110 \n\nI- Pulse  Length \n\n~ \n\n(a) \n\n-QJ-\n\n(b) \n\nFigure  2.  (a).  The  neuron  spike-generating  circuit.  (b).  The  follower-integrater \ncircuit.  Each  delay  box 0 contains  a  delay line  formed  by  three follower-integrater \ncircuits. \n\nThe Endogenous Bursting Neuron \n\nOne  type  of cell  which  has  been  found  to  play  an important  role  in  many  oscilla(cid:173)\ntory circuits  is  the  endogenous  bursting  neuron.  This type  of cell  has  an  intrinsic \noscillatory  membrane  potential, enabling  it  to  produce bursts of  action  potentials \nat rhythmic  intervals.  These  cells  have  been shown to act  both  as  external  \"pace(cid:173)\nmakers\"  which  set  the  rhythm  for  the  CPG,  or  as  an  integral  part  of  a  central \npattern generator.  Figure  3a shows the output from  a  biological endogenous burst(cid:173)\ning  neuron.  Figure  3b  demonstrates  how  we  can  configure  our  CPG  neuron to be \nan endogenous bursting neuron.  The delay element in the cell must have three time \nconstants in  order for  this  circuit  to  oscillate  stably.  Note  that  in  the  circuit,  the \n\n\fModeling Small Oscillating Biological Networks \n\n387 \n\ncell has internal negative feedback.  Since real neurons don't actually make synaptic \nconnections  onto themselves,  this  connection should  be  thought  of  as  representing \nan internal molecular or ionic  mechanism which results in feedback  within the cell. \n\nAS \n\n(a) \n\n4  mV \n\n1  sec \n\n(b) \n\n(c) \n\nFigure 3.  (a).  The output from  the AB  cell in the lobster stomatogastric ganglion \nCPG  [Eisen  and  Marder,  1982].  This  cell  is  known  to  burst  endogenously.  (b). \nThe  CPG  neuron  circuit  configured  as  an endogenous bursting neuron and  (c)  the \noutput from  this circuit. \n\nPostinhibitory Rebound \n\nA  neuron  configured  to  be  an  endogenous  burster  also  exhibits  another  property \ncommon to many neurons,  including many CPG neurons.  This property, illustrated \nin Figures 4a and 4b,  is  known as  postinhibitory rebound  (PIR). Neurons with this \nproperty  display  increased  excitation  for  a  certain  period  of  time  following  the \nrelease  of an inhibitory influence.  This  property  is  a  useful one for  central pattern \ngenerator  neurons  to  have,  because  it  enables  patterns  of  oscillations  to  be  reset \nfollowing  the release  of inhibition. \n\n\f388 \n\nRyckebusch, Bower and Mead \n\n-(a) \n\n.. I \n\n.. \n\n\" ... \n\n\u2022 \n\n' \n\nLA \n\n4 \n\nt  \u2022\u2022\u2022\u2022 \n\n(c) \n\n(b) \n\nN' \n\nI \nI  ........ \n\nFigure  4.  (a)  The  output  of  a  ganglion  cell  of  the  mudpuppy  retina  exhibiting \npostinhibitory rebound  [Miller  and  Dacheux,  19761.  The bar under the  trace indi(cid:173)\ncates the duration of the inhibition.  (b)  To exhibit  PIR in  the CPG neuron circuit, \nwe  inhibit ,the  cell  with  the  square  pulse  shown  in  (c).  When  the  inhibition  is \nreleased,  the circuit outputs a  brief burst of pulses. \n\nMODELING  CENTRAL PATTERN  GENERATORS \n\nThe Lobster Stomatogastric  Ganglion \n\nThe  stomatogastric  ganglion  is  a  CPG  which  controls  the  movement  of the  teeth \nin  the  lobster's  stomach.  This  network  is  relatively  complex,  and  we  have  only \nmodeled the relationships between two of the neurons  in  the  CPG  (the  PD and LP \ncells)  which  have  a  kind  of interaction  found  in  many  CPGs  known  as  reciprocal \ninhibition  (Figure  Sa).  In  this case,  each  cell  inhibits  the  other,  which  produces  a \npattern of output in which the cells  fire  alternatively  (Figure Sb).  Note  that  in  the \nabsence  of external input,  a  mechanism such  as  postinhibitory rebound must exist \nin order for  a cell to begin firing  again once  it  has  been released  from  inhibition. \n\n\fModeling Small Oscillating Biological Networks \n\n389 \n\n120 \n_ \n~rrN \n\n(a) \n\n(b) \n\n(c) \n\n(d) \n\nFigure  5.  (a)  Output  from  the  PD  and  LP  cells  in  the  lobster  stomatogastric \nganglion [Miller and Selverston, 1985].  (c)  and (d) demonstrate reciprocal inhibition \nwith two  CPG neuron circuits. \n\nThe Locust Flight  CPG \n\nA  CPG has  been shown  to play  an important role  in producing the motor pattern \nfor  flight  in the locust  [Robertson and Pearson,  19851.  Two of the cells  in  the CPG, \nthe  301  and  501  cells,  fire  bursts of  action  potentials  as  shown  in  Figure  6a.  The \n301  cell is  active  when  the wings  of the  locust  are elevated,  whereas  the  501  cell  is \nactive  when  the wings  are  depressed.  The phase relationship  between  the two cells \nis  very  similar  to  the  reciprocal  inhibition  pattern just  discussed,  but  the  circuit \nthat  produces  this  pattern  is  quite  different.  The  connections  between  these  two \ncells  are  shown  in  Figure  6b.  The  301  cell  makes  a  delayed  excitatory  connection \nonto the  501  cell,  and  the  501  cell makes fast  inhibitory contact  with the  301  cell. \nTherefore, the 301  cell begins to fire,  and after some delay,  the 501  cell is  activated. \nWhen the 501  cell begins to fire,  it immediately shuts off the 301 cell.  Since the 501 \ncell  is  no  longer receiving  excitatory  input,  it will  eventually  stop  firing,  releasing \nthe  301  cell  from  inhibition.  The  cycle  then repeats.  This  same  circuit  has  been \nreproduced with our model in  Figures 6c  and 6d. \n\n\f390 \n\nRyckebusch, Bower and Mead \n\n\u2022 \n\n(a) \n\n~ \n\\5'~'1 \n\nI--~  501 \n\n---.J50mv \n100ms \n\n301~~~~=-~~~~~~ __ ~ __ ~~ \nDl \n\n(b) \n\n(d) \n\n301 \n\n301  CELL \n\nSOl  CELL \n\n(e) \n\nFigure  6. \nPearson,  19851. \nflight.  (c)  The model circuit  and  (d)  its output. \n\n(a)  The  301  and  501  cells  in  the  locust  flight  CPG  [Robertson  and \n(b)  Simultaneous  intracellular  recordings  of  301  and  501  during \n\nThe  Tritonia Swim CPG \n\nOne  of the  best  studied  central pattern  generators  is  the  CPG  which  controls  the \nswimming  in  the  small marine  mollusc  Tritonia.  This  CPG  was  studied  in  great \ndetail by  Peter  Getting  and  his  colleagues  at  the  University  of  Iowa,  and it  is  one \nof  the  few  biological  neural  networks  for  which  most  of  the  connections  and  the \nsynaptic  parameters  are  known  in  detail.  Tritonia  swims  by  making  alternating \ndorsal  and  ventral flexions.  The dorsal  and ventral  motor  neurons  are  innervated \nby  the  DSI  and  VSI cells,  respectively.  Shown  in  Figure  7a and  7b  is  a  simplified \nschematic diagram for the network and the corresponding output.  The DSI and VSI \ncells fire  out of phase, which is  consistent with the alternating nature of the animal's \nswimming motion.  The  basic  circuit  consists  of reciprocal  inhibition  between  DSI \nand  VSI  paralleled  by  delayed  excitation  via  the  C2  cell.  The  DSI  and  VSI  cells \nfire  out of phase,  and the DSI  and  C2  cells  fire  in  phase.  Swimming is  initiated by \nsensory stimuli which feed  into DSI and cause it to begin to fire  a burst of impulses. \nDSI  inhibits  VSI,  and  at  the  same  time  excites  C2.  C2  has  excitatory  synapses \non  VSIj  however,  the  initial  response  of  VSI  neurons  is  delayed.  VSI  then  fires, \nduring which there is  inhibition by VSI of C2  and  DSI.  During this  period,  VSI no \nlonger  receives  excitatory  input  from  C2,  and  hence  the  VSI  firing  rate  declines; \nDSI  is  therefore  released  from  inhibition,  and  is  ready  to  fire  again  to  initiate  a \nnew  cycle.  Figure 7c  and 8 show the  model circuit which  is  identical to the  circuit \n\n\fModeling Small Oscillating Biological Networks \n\n391 \n\nshown in  Figure 7a,  and the output from this circuit.  Note that although the model \noutput closely resembles the biological data, there are small differences in the phase \nrelationships  between  the  cells which  can be accounted  for  by taking into account \nother connections and delays in the circuit not currently incorporated in our model. \n\n-\n\nOSI  : I \n\nVSI \nB \n\nC2 \n\n(b) \n\nC2 \n\n50  mV J \n\n5  sec \n\n.' .~ \n\n.' \n\n.' \n\n........... \n\n. - - --I \n\n(a) \n\n.. \n\n1-0 \n\n.. .. \n..... \n..... \n\n\\ \n\\ \n\\ \n\\ \n\\ \n\\ \n\nDSI \n\n.....  , \n...... \\ \nt' ... ,  ......  ' \n, , , \n\\ ,----------\n\nVSI \n\n(c) \n\nFigure  1.  (a)  Simplified schematic  diagram  of the  Tritonia  CPG  (which  actually \nhas  14  cells)  and  (b)  output  from  the  three  types  of  cells  in  the  circuit.(c)  The \nmodel circuit. \n\n\f392 \n\nRyckebusch, Bower and Mead \n\nVSI \n\nC2 \n\nDSI \n\nFigure 8.  Output from  the circuit  shown  in  Figure 7c. \n\nCONCLUSIONS \n\nOne  may  ask  why  it is  interesting  to model  these  systems  in  analog  VLSI,  or,  for \nthat matter, why it is interesting to model invertebrate networks altogether.  Analog \nVLSI is  a very  nice medium for  this type of modeling, because in addition to being \ncompact, it runs in real time, eliminating the need to wait hours to get the results of \na simulation.  In addition, the electronic circuits rely on the same physical principles \nas neural processes {including gain, delays,  and feedback},  allowing us to exploit the \ninherent properties of the medium in which we work rather than having to explicitly \nmodel them as  in  a  digital simulation. \n\nLike  all  models,  we  hope  that  this  work  will  help  us  learn  something  about  the \nsystems we  are studying.  But in addition, although invertebrate neural networks are \nrelatively  simple  and  have  small numbers of cells,  the behaviours of these networks \nand  animals  can be  fairly  complex.  At  the same time,  their small size  allows  us  to \nunderstand how they are engineered in detail.  Accordingly, modeling these networks \nallows  us  to  study  a  well  engineered  system  at  the  component  level-a  level  of \nmodeling not yet possible for more complex mammalian systems, for which detailed \nstructural information is  scarce. \n\n\fModeling Small Oscillating Biological Networks \n\n393 \n\nAcknowledgments \n\nThis work relies on information supplied by the hard work of many experimentalists. \nWe  would  especially  like  to acknowledge  the effort  and dedication of Peter Getting \nwho devoted 12 years to understanding the organization of the Tritonia network  of \n14  neurons.  We  also  thank  Hewlett-Packard  for  computing  support,  and  DARPA \nand  MOSIS  for  chip  fabrication.  This  work  was  sponsored  by  the  Office  of  Naval \nResearch,  the  System  Development  Foundation,  and  the  NSF  (EET-8700064  to \nJ.B.). \n\nReference8 \n\nEisen,  Judith  S.  and  Marder,  Eve  (1982).  Mechanisms  underlying  pattern  gener(cid:173)\nation  in  lobster stomatogastric ganglion  as  determined  by  selective  inactivation of \nidentified neurons.  III. Synaptic connections of electrically coupled pyloric neurons. \nJ.  Neurophysiol.  48:1392-1415. \n\nGetting,  Peter  A.  and  Dekin,  Michael  S.  (1985).  Tritonia  swimming:  A  model \nsystem for integration within rhythmic motor systems.  In Allen I. Selverston  (Ed.), \nModel  Neural  Networks  and  Behavior,  New  York,  NY:  Plenum  Press. \nMead,  Carver  A.  (in  press).  Analog  VLSI  and  Neural  Systems.  Reading,  MA: \nAddison-Wesley. \nMiller,  John P.  and Selverston,  Allen I.  (1985).  Neural Mechanisms for  the produc(cid:173)\ntion of the lobster pyloric motor pattern.  In Allen I. Selverston  (Ed.), Model Neural \nNetworks  and Behavior,  New  York,  NY:  Plenum  Press. \n\nMiller,  R.  F.  and  Dacheux,  R.  F.  (1976).  Synaptic organization and ionic  basis  of \non  and  off channels in  mudpuppy retina.  J.  Gen.  Physiol.  67:639-690. \n\nRobertson,  R.  M.  and  Pearson,  K.  G.  (1985).  Neural  circuits  in  the  ft.ight  system \nof the  locust.  J.  Neurophysiol.  53:110-128. \n\nSelverston,  Allen  I.  and  Moulins,  Maurice  (1985).  Oscillatory  neural  networks. \nAnn.  Rev.  Physiol.  47:29-48. \nWilson, M.  and Bower,  J.  M.  (in press).  Simulation oflarge scale neuronal networks. \nIn  C.  Koch  and I.  Segev  (Eds.),  Methods  in  Neuronal  Modeling:  From  Synapses  to \nNetworks,  Cambridge j  MA:  MIT Press. \n\n\f", "award": [], "sourceid": 131, "authors": [{"given_name": "Sylvie", "family_name": "Ryckebusch", "institution": null}, {"given_name": "James", "family_name": "Bower", "institution": null}, {"given_name": "Carver", "family_name": "Mead", "institution": null}]}