{"title": "A Method for the Design of Stable Lateral Inhibition Networks that is Robust in the Presence of Circuit Parasitics", "book": "Neural Information Processing Systems", "page_first": 860, "page_last": 868, "abstract": null, "full_text": "860 \n\nA  METHOD  FOR  THE  DESIGN  OF  STABLE  LATERAL  INHIBITION \n\nNETWORKS  THAT  IS  ROBUST  IN  THE  PRESENCE \n\nOF  CIRCUIT  PARASITICS \n\nJ.L.  WYATT,  Jr  and  D.L.  STANDLEY \n\nDepartment  of  Electrical  Engineering  and  Computer  Science \n\nMassachusetts  Institute  of  Technology \n\nCambridge,  Massachusetts  02139 \n\nABSTRACT \n\nIn  the  analog  VLSI  implementation  of  neural  systems,  it is \n\nsometimes  convenient  to  build  lateral  inhibition  networks  by  using \na  locally  connected  on-chip  resistive  grid.  A  serious  problem \nof  unwanted  spontaneous  oscillation  often  arises  with  these \ncircuits  and  renders  them  unusable  in  practice.  This  paper  reports \na  design  approach  that  guarantees  such  a  system  will  be  stable, \neven  though  the  values  of  designed  elements  and  parasitic  elements \nin  the  resistive  grid  may  be  unknown.  The  method  is  based  on  a \nrigorous,  somewhat  novel  mathematical  analysis  using  Tellegen's \ntheorem  and  the  idea  of  Popov  multipliers  from  control  theory.  It \nis  thoroughly  practical  because  the  criteria  are  local  in  the  sense \nthat  no  overall  analysis  of  the  interconnected  system  is  required, \nempirical  in  the  sense  that  they  involve  only  measurable  frequency \nresponse  data  on  the  individual  cells,  and  robust  in  the  sense  that \nunmodelled  parasitic  resistances  and  capacitances  in  the  inter(cid:173)\nconnection  network  cannot  affect  the  analysis. \n\nI. \n\nINTRODUCTION \n\nThe  term  \"lateral  inhibition\"  first  arose  in  neurophysiology  to \n\ndescribe  a  common  form  of  neural  circuitry  in  which  the  output  of \neach  neuron  in  some  population  is  used  to  inhibit  the  response  of \neach  of  its  neighbors.  Perhaps  the  best  understood  example  is  the \nhorizontal  cell  layer  in  the  vertebrate  retina,  in  which  lateral \ninhibition  simultaneously  enhances  intensity  edges  and  acts  as  an \nautomatic  lain  control  to  extend  the  dynamic  range  of  the  retina \nas  a  whole.  The  principle  has  been  used  in  the  design  of  artificial \nneural  system  algorithms  by  Kohonen2  and  others  and  in  the  electronic \ndesign  of  neural  chips  by  Carver  Mead  et.  al. 3 ,4. \n\nIn  the  VLSI  implementation  of  neural  systems,  it is  convenient \n\nto  build  lateral  inhibition  networks  by  using  a  locally  connected \non-chip  resistive  grid.  Linear  resistors  fabricated  in,  e.g., \npolysilicon,  yield  a  very  compact  realization,  and  nonlinear \nresistive  grids,  made  from  MOS  transistors,  have  been  found  useful \nfor  image  segmentation. 4 ,5  Networks  of  this  type  can  be  divided  into \ntwo  classes:  feedback  systems  and  feedforward-only  systems. \nfeedforward  case  one  set  of  amplifiers  imposes  signal  voltages  or \n\nIn  the \n\n\u00a9 American Institute of Physics 1988 \n\n\f861 \n\ncurrents  on  the  grid  and  another  set  reads  out  the  resulting  response \nfor  subsequent  processing,  while  the  same  amplifiers  both  \"write\"  to \nthe  grid  and  \"read\"  from  it in  a  feedback  arrangement.  Feedforward \nnetworks  of  this  type  are  inherently  stable,  but  feedback  networks \nneed  not  be. \n\nA  practical  example  is  one  of  Carver  Meadls  retina  chips3  that \n\nachieves  edge  enhancement  by  means  of  lateral  inhibition  through  a \nresistive  grid.  Figure  1  shows  a  single  cell  in  a  continuous-time \nversion  of  this  chip.  Note  that  the  capacitor  voltage  is  affected \nboth  by  the  local  light  intensity  incident  on  that  cell  and  by  the \ncapacitor  voltages  on  neighboring  cells  of  identical  design.  Any \ncell  drives  its  neighbors,  which  drive  both  their  distant  neighbors \nand  the  original  cell  in  turn.  Thus  the  necessary  ingredients  for \ninstability--active  elements  and  signal  feedback--are  both  present \nin  this  system,  and  in  fact  the  continuous-time  version  oscillates \nso  badly  that  the  original  design  is  scarcely  usable  in  practice \nwith  the  lateral  inhibition  paths  enabled. 6  Such  oscillations  can \n\nincident \nlight \n\nI \n\nv \nout \n\nFigure  1.  This  photoreceptor  and  signal  processor  Circuit,  using  two \nMOS  transconductance  amplifiers,  realizes  lateral  inhibition  by \ncommunicating  with  similar  units  through  a  resistive  grid. \n\nreadily  occur  in  any  resistive  grid  circuit  with  active  elements  and \nfeedback,even  when  each  individual  cell  is  quite  stable.  Analysis \nof  the  conditions  of  instability  by  straightforward  methods  appears \nhopeless,  since  any  repeated  array  contains  many  cells,  each  of \nwhich  influences  many  others  directly  or  indirectly  and  is  influenced \nby  them  in  turn,  so  that  the  number  of  simultaneously  active  feed(cid:173)\nback  loops  is  enormous. \n\nThis  paper  reports  a  practical  design  approach  that  rigorously \nguarantees  such  a  system  will  be  stable.  The  very  simplest  version \nof  the  idea  is  intuitively  obvious:  design  each  individual  cell  so \nthat,  although  internally  active,  it acts  like  a  passive  system  as \nseen  from  the  resistive  grid. \ndesign  goal  here  is  that  each  cellis  output  impedance  should  be  a \npositive-real?  function.  This  is  sometimes  not  too  difficult  in \npractice;  we  will  show  that  the  original  network  in  Fig.  1  satisfies \nthis  condition  in  the  absence  of  certain  parasitic  elements.  More \nimportant,  perhaps,  it  is  a  condition  one  can  verify  experimentally \n\nIn  circuit  theory  language,  the \n\n\f862 \n\nby  frequency-response  measurements. \n\nIt  is  physically  apparent  that  a  collection  of  cells  that \n\nappear  passive  at  their  terminals  will  form  a  stable  system  when \ninterconnected  through  a  passive  medium  such  as  a  resistive  grid. \nThe  research  contributions,  reported  here  in  summary  form,  are \ni)  a  demonstration  that  this  passivity  or  positive-real  condition \nis  much  stronger  than  we  actually  need  and  that  weaker  conditions, \nmore  easily  achieved  in  practice,  suffice  to  guarantee  stability  of \nthe  linear  network  model,  and  ii)  an  extension  of  i)  to  the  nonlinear \ndomain  that  furthermore  rules  out  large-signal  oscillations  under \ncertain  conditions. \n\nII.  FIRST-ORDER  LINEAR  ANALYSIS  OF  A  SINGLE  CELL \n\nWe  begin  with  a  linear  analysis  of  an  elementary  model  for  the \n\ncircuit  in  Fig.  1.  For  an  initial  approximation  to  the  output \nadmittance  of  the  cell  we  simplify  the  topology  (without  loss  of \nrelevant  information)  and  use  a  naive'model  for  the  transconductance \namplifiers,  as  shown  in  Fig.  2. \n\ne \n+ \n\nFigure  2.  Simplified  network  topology  and  transconductance  amplifier \nmodel  for  the  circuit  in  Fig.  1.  The  capacitor  in  Fig.  1  has  been \nabsorbed  into  CO2 \u2022 \n\nStraightforward  calculations  show  that  the  output  admittance  is \n\ngiven  by \n\nyes) \n\n(1) \n\nThis  is  a  positive-real,  i.e.,  passive,  admittance  since  it can  always \nbe  realized  by  a  network  of  the  form  shown  in  Fig.  3,  where \nRl  =  (gm2+  Ro2 ) \n\n,  and  L  =  COI/gmlgm2\u00b7 \n\n-1  -1 \n\n,  R2=  (gmlgm2Rol) \n\n-1 \n\nAlthough  the  original  circuit  contains  no  inductors,  the \n\nrealization  has  both  capacitors  and  inductors  and  thus  is  capable \nof  damped  oscillations.  Nonetheless,  if the  transamp  model  in \nFig.  2  were  perfectly  accurate,  no  network  created  by  interconnecting \nsuch  cells  through  a  resistive  grid  (with  parasitic  capacitances) \ncould  exhibit  sustained  oscillations.  For  element  values  that  may \nbe  typical  in  practice,  the  model  in  Fig.  3  has  a  lightly  damped \nresonance  around  I  KHz  with  a  Q  ~  10.  This  disturbingly  high  Q \nsuggests  that  the  cell  will  be  highly  sensitive  to  parasitic  elements \nnot  captured  by  the  simple  models  in  Fig.  2.  Our  preliminary \n\n\f863 \n\nyes) \n\nFigure  3.  Passive  network  realization  of  the  output  admittance  (eq. \n(1)  of  the  circuit  in  Fig.  2. \n\nanalysis  of  a  much  more  complex  model  extracted  from  a  physical \ncircuit  layout  created  in  Carver  Mead's  laboratory  indicates  that \nthe  output  impedance  will  not  be  passive  for  all  values  of  the  trans(cid:173)\namp  bias  currents.  But  a  definite  explanation  of  the  instability \nawaits  a  more  careful  circuit modelling  effort  and  perhaps  the  design \nof  an  on-chip  impedance  measuring  instrument. \n\nIII.  POSITIVE-REAL  FUNCTIONS,  e-POSITlVE  FUNCTIONS,  AND \n\nSTABILITY  OF  LINEAR  NETWORK  MODELS \n\nIn  the  following  discussion  s  =  cr+jw  is  a  complex  variable, \nH(s)  is  a  rational  function  (ratio  of  polynomials)  in  s  with  real \ncoefficients,  and  we  assume  for  simplicity  that  H(s)  has  no  pure \nimaginary  poles.  The  term  closed  right  halE  plane  refers  to  the  set \nof  complex  numbers  s  with  Re{s}  >  o. \n\nDef.  I \n\nThe  function  H(s)  is  said  to  be  positive-real  if a)  it has  no \n\npoles  in  the  right  half  plane  and  b)  Re{H(jw)}  ~ 0  for  all  w. \n\nIf we  know  at  the  outset  that  H(s)  has  no  right  half  plane  poles, \n\nthen  Def.  I  reduces  to  a  simple  graphical  criterion:  H1s}  is  positive(cid:173)\nreal  if  and  only  if  the  Nyquist  diagram  of  H(s)  (i.e.  the  plot  of \nH(jW)  for  w ~ 0,  as  in  Fig.  4)  lies  entirely  in  the  closed  right  half \nplane. \n\nNote  that positive-real  functions  are  necessarily  stable  since \n\nthey  have  no  right  half  plane  poles,  but  stable  functions  are  not \nnecessarily  positive-real,  as  Example  1  will  show. \n\nA  deep  link  between  positive  real  functions,  physical  networks \n\nand  passivity  is  established  by  the  classical  result7  in  linear \ncircuit  theory  which  states  that  H(s)  is  positive-real  if  and  only  if \nit  is  possible  to  synthesize  a  2-terminal  network  of  positive  linear \nresistors,  capacitors,  inductors  and  ideal  transformers  that  has  H(s) \nas  its  driving-point  impedance  or  admittance. \n\n\f864 \n\nOef.  2 \n\nThe  function  H(s)  is  said  to  be  a-positive  for  a  particular  value \n\nof  e(e  ~ 0,  e  ~ ~),  if  a)  H{s)  has  no  poles  in  the  right  half  plane, \nand  b)  the  Nyquist  plot  of  H(s)  lies  strictly  to  the  right  of  the \nstraight  line passing  through  the  origin  at  an  angle  a  to  the  real \npositive  axis. \n\nNote  that  every  a-positive  function  is  stable  and  any  function \n\nthat  is  e-positive  with  e  = ~/2  is  necessarily  positive-real. \n\nI  {G(jw)} \nm \n\nRe{G(jw) } \n\nFigure  4.  Nyquist  diagram  for  a  fUnction  that  is  a-positive  but \nnot  positive-real. \n\nExample  1 \n\nThe  function \n\n(s+l) (s+40) \nG (s)  =  (s+5)  (s+6)  (s+7) \n\n(2) \n\nis  a-positive  (for  any  e  between  about  18\u00b0  and  68\u00b0)  and  stable, but  it \nis  not  positive-real  since  its  Nyquist  diagram,  shown  in  Fig.  4, \ncrosses  into  the  left half  plane. \n\nThe  importance  of  e-positive  functions  lies  in  the  following \n\nobservations:  1)  an  interconnection  of  passive  linear  resistors  and \ncapacitors  and  cells  with  stable  linear  impedances  can  result  in  an \nunstable  network,  b)  such  an  instability  cannot  result  if  the \nimpedances are  also  positive-real,  c)  a-positive  impedances  form  a \nlarger  class  than  positive-real  ones  and  hence  a-positivity  is  a  less \ndemanding  synthesis  goal,  and  d)  Theorem  1  below  shows  that  such  an \ninstability  cannot  result  if  the  impedances  are  a-positive, even  if \nthey  are  not  positive-real. \n\nTheorem  1 \n\nConsider  a  linear  network  of  arbitrary  topology,  consisting  of \n\nany  number  of  passive  2-terminal  resistors  and  capacitors  of  arbitrary \nvalue  driven  by  any  number  of  active  cells.  If  the  output  impedances \n\n\f865 \n\n'II\" \nof  all  the  active  cells  are  a-positive  for  some  common  a,  0<a22, \nthen  the  network  is  stable. \n\nThe  proof  of  Theorem  1  relies  on  Lemma  1  below. \n\nLemma  1 \n\nIf  H(s)  is  a-positive  for  some  fixed  a,  then  for  all  So  in  the \nclosed  first  quadrant  of  the  complex  plane,  H(so)  lies  strictly  to \nthe  right  of  the  straight  line  passing  through  the  origin  at  an  angle \na  to  the  real  positive  axis,  i.e.,  Re{so}  ~ 0  and  Im{so}  ~ 0  ~ \na-'II\"  < L  H (so)  <  a. \n\nProof  of  Lemma  1  (Outline) \n\nLet  d  be  the  function  that  assigns  to  each  s  in  the  closed  right \n\nhalf  plane  the  perpendicular  distance  des)  from  H(s)  to  the  line \ndefined  in  Def.  2.  Note  that  des)  is  harmonic  in  the  closed  right \nhalf  plane,  since  H  is  analytic  there.  It  then  follows,  by  application \nof  the  maximum  modulus  principle8  for  harmonic  functions,  that  d  takes \nits minimum  value  on  the  boundary  of  its  domain,  which  is  the \nimaginary  axis.  This  establishes  Lemma  1. \n\nProof  of  Theorem  1  (OUtline) \n\nThe  network  is  unstable  or  marginally  stable  if  and  only  if  it \nhas  a  natural  frequency  in  the  closed  right half  plane,  and  So  is  a \nnatural  frequency  if  and  only  if  the  network  equations  have  a  nonzero \nsolution  at  so.  Let  {Ik}  denote  the  complex  branch  currents  Of  such \na  solution.  By  Tellegen I  s  theorern9 the  sum  of  the  complex  powers \nabsorbed  by  the  circuit  elements  must  vanish  at  such  a  solution,  i.e., \n\n~ \n\nIIk12/s0Ck  + \n\ncapac~tances \n\nL \ncell \n\nterminal  pairs \n\n(3) \nwhere  the  second  term  is  deleted  in  the  special  case  so=O,  since  the \ncomplex  power  into  capacitors  vanishes  at  so=O. \n\nIf  the  network  has  a  natural  frequency  in  the  closed  right  half \n\nplane,  it must  have  one  in  the  closed  first  quadrant  since  natural \nfrequencies  are  either  real  or  else  occur  in  complex  conjugate  pairs. \nBut  (3)  cannot  be  satisfied  for  any  So  in  the  closed  first  quadrant, \n\nas  we  can  see  by  dividing  both  sides  of  (3)  by  k IIkI2,  where  the \n\nsum  is  taken  over  all  network  branches.  After  this  division,  (3) \nasserts  that  zero  is  a  convex  combination  of  terms  of  the  form  Rk, \nterms  of  the  form  (CkSo)-I,  and  terms  of  the  form  Zk(So). \nVisualize  where  these  terms  lie  in  the  complex  plane:  the  first  set  lies \non  the  real  positive  axis,  the  second  set  lies  in  the  closed  4-th \n~adrant since  So  lies  in  the  closed  1st  quadrant  by  assumption,  and \nthe  third  set  lies  to  the  right  of  a  line  passing  through  the  origin \nat  an  angle  a  by  Lemma  1.  Thus  all  these  terms  lie  strictly  to  the \nright  of  this  line,  which  implies  that  no  convex  combination  of  them \ncan  equal  zero.  Hence  the  network  is  stable! \n\n\f866 \n\nIV.  STABILITY  RESULT  FOR  NETWORKS  WITH  NONLINEAR \n\nRESISTORS  AND  CAPACITORS \n\nThe  previous  result  for  linear  networks  can  afford  some  limited \ninsight  into  the  behavior  of  nonlinear  networks.  First  the  nonlinear \nequations  are  linearized  about  an  equilibrium  point  and  Theorem  1  is \napplied  to  the  linear  model. \nIf  the  linearized  model  is  stable,  then \nthe  equilibrium  point  of  the  original  nonlinear  network  is  locally \nstable,  i.e.,  the  network  will  return  to  that  equilibrium  point  if \nthe  initial  condition  is  sufficiently  near  it.  But  the  result  in  this \nsection,  in  contrast,  applies  to  the  full  nonlinear  circuit  model  and \nallows  one  to  conclude  that  in  certain  circumstances  the  network \ncannot  oscillate  even  if the  initial  state  is  arbitrarily  far  from \nthe  equilibrium  point. \n\nDef.  3 \n\nA  function  H(s)  as  described  in  Section  III  is  said  tc  satisfy \n\nthe  Popov  criterionlO  if  there  exists  a  real  number  r>O  such  that \nRe{(l+jwr)  H(jw)}  ~ 0  for  all  w. \n\nNote  that  positive  real  functions  satisfy  the  Popov  criterion \nwith  r=O.  And  the  reader  can  easily  verify  that  G(s)  in  Exam~le I \nsatisfies  the  Popov  criterion  for  a  range  of  values  of  r.  The  important \neffect  of  the  term  (l+jwr)  in  Def.  3  is  to  rotate  the  Nyquist  plot \ncounterclockwise  by  progressively  greater  amounts  up  to  90\u00b0  as  w \nincreases. \n\nTheorem  2 \n\nConsider  a  network  consisting  of  nonlinear  2-terminal  resistors \n\nand  capacitors,  and  cells  with  linear  output  impedances  ~(s).  Suppose \n\ni)  the  resistor  curves  are  characterized  by  continuously \ndiffefentiable  functions  i k  =  gk(vk )  where  gk(O)  =  0 \nand \no  <  gk(vk )  <  G  <  00  for  all  values  of  k  and  vk' \nii)  the  capacitors  are  characterized  by  i k  = Ck(Vk)~k with \no  <  CI  <  Ck(vk )  <  C2  <  00  for  all  values  of  k  and  vk' \niii)  the  impedances  Zk(s)  have  no  poles  in  the  closed  right \nhalf  plane  and  all  satisfy  the  Popov  criterion  for  some  common \nvalue  of  r. \n\nIf  these  conditions  are  satisfied,  then  the  network  is  stable  in  the \nsense  that,  for  any  initial  condition, \n\nfoo( \no  all  branches \n\nI \n\ni~(t)  dt \n\n) \n\n<  00 \n\n\u2022 \n\n(4) \n\nThe  proof,  based  on  Tellegen's  theorem,  is  rather  involved.  It \n\nwill  be  omitted  here  and  will  appear  elsewhere. \n\n\f867 \n\nACKNOWLEDGEMENT \n\nWe  sincerely  thank  Professor  Carver  Mead  of  Cal  Tech  for \n\nenthusiastically  supporting  this  work  and  for  making  it possible  for \nus  to  present  an  early  report  on  it  in  this  conference  proceedings. \nThis  work  was  supportedJ::\u00a5 Defense  Advanced  Research  Projects  Agency \n(DoD),  through  the  Office  of  Naval  Research  under  ARPA  Order  No. \n3872,  Contract  No.  N00014-80-C-0622  and  Defense  Advanced  Research \nProjects  Agency  (DARPA)  Contract  No.  N00014-87-R-0825. \n\nREFERENCES \n\n1.  F.S.  Werblin,  \"The  Control  of  Sensitivity  on  the  Retina,\" \n\nScientific  American,  Vol.  228,  no.  1,  Jan.  1983,  pp.  70-79. \n\n2.  T.  Kohonen,  Self-Organization  and  Associative  Memory,  (vol.  8  in \n\nthe  Springer  Series  in  Information  Sciences),  Springer  Verlag, \nNew  York,  1984. \n\n3.  M.A.  Sivilotti,  M.A.  Mahowald,  and  C.A.  Mead,  \"Real  Time  Visual \n\nComputations  Using  Analog  CMOS  processing  Arrays,\"  Advanced \nResearch  in  VLSI  - Proceedings  of  the  1987  Stanford  Conference, \nP.  Losleben,  ed.,  MIT  Press,  1987,  pp.  295-312. \n\n4.  C.A.  Mead,  Analog  VLSI  and  Neural  Systems,  Addison-Wesley,  to \n\nappear  in  1988. \n\n5.  J.  Hutchinson,  C.  Koch,  J.  Luo  and  C.  Mead,  \"Computing  Motion \nUsing  Analog  and  Binary  Resistive  Networks,\"  submitted  to  IEEE \nTransactions  on  Computers,  August  1987. \n\n6.  M.  Mahowald,  personal  communication. \n7.  B.D.O.  Anderson  and  S.  Vongpanitlerd,  Network  Analysis  and \n\nsynthesis  - A  Modern  Systems  Theory  Approach,  Prentice-Hall, \nEnglewood  Cliffs,  NJ.,  1973. \n\n8.  L.V.  Ahlfors,  Complex  Analysis,  McGraw-Hill,  New  York,  1966, \n\np.  164. \n\n9.  P.  penfield,  Jr.,  R.  Spence,  and  S.  Duinker,  Tellegen's  Theorem \n\nand  Electrical  Networks,  MIT  Press,  Cambridge,  MA,1970. \n\n10.  M.  Vidyasagar,  Nonlinear  Systems  Analysis,  Prentice-Hall, \n\nEnglewood  Cliffs,  NJ,  1970,  pp.  211-217. \n\n\f\f", "award": [], "sourceid": 54, "authors": [{"given_name": "John", "family_name": "Wyatt", "institution": null}, {"given_name": "D.", "family_name": "Standley", "institution": null}]}