{"title": "Visual Motion Computation in Analog VLSI Using Pulses", "book": "Advances in Neural Information Processing Systems", "page_first": 781, "page_last": 788, "abstract": null, "full_text": "Visual Motion  Computation in  Analog \n\nVLSI using Pulses \n\nRahul  Sarpeshkar, Wyeth Bair and  Christof Koch \n\nComputation and Neural Systems Program \n\nCalifornia Institute of Technology \n\nPasadena, CA 91125. \n\nAbstract \n\nThe  real  time  computation  of  motion  from  real  images \nusing a  single chip with integrated sensors is a  hard prob(cid:173)\nlem.  We  present two  analog VLSI schemes that use  pulse \ndomain neuromorphic circuits to compute motion.  Pulses \nof variable width, rather than graded potentials, represent \na  natural  medium  for  evaluating temporal relationships. \nBoth algorithms measure speed by  timing a  moving edge \nin  the image.  Our first  model is  inspired  by  Reichardt's \nalgorithm in the fiy  and yields  a  non-monotonic response \nvs.  velocity  curve.  We  present  data from  a  chip  that \nimplements  this  model.  Our  second  algorithm  yields  a \nmonotonic  response  vs.  velocity  curve  and  is  currently \nbeing translated into silicon. \n\n1 \n\nIntrod uction \n\nAnalog  VLSI  chips  for  the  real  time  computation of visual  motion  have  been  the \nfocus  of much  active  research  because  of their  importance as  sensors  for  robotic \napplications.  Correlation  schemes  such  as  those  described  in \n(Delbriick,  1993) \nhave been found to be more robust than gradient schemes described in  (Tanner and \nMead,  1986),  because  they  do  not  involve  noise-sensitive  operations like  spatial(cid:173)\ndifferentiation  and  division.  A  comparison  of four  experimental  schemes  may  be \nfound in  (Horiuchi et al.,  1992).  In spite of years of work,  however, there is still  no \nmotion chip that robustly computes motion under all  environmental conditions. \n\n781 \n\n\f782 \n\nSarpeshkar,  Bair,  and Koch \n\nMotion  algorithms  operating on  higher  level  percepts  in  an  image  such  as  zero(cid:173)\ncrossings (edges) are more robust than those that operating on lower level percepts \nin an image such as raw image intensity values  (Marr and Ullman,  1981).  Our work \ndemonstrates how,  if the edges ill  an image are identified, it is possible to compute \nmotion, quickly  and easily,  by  using pulses.  We compute the velocity at each  point \nin  the  image.  The  estimation  of  the  flow-field  is  of  tremendous  importance  in \ncomputations such  as  time-to-contact,  figure-ground-segregation  and  depth-from(cid:173)\nmotion.  Our motion scheme is well-suited to typical indoor environments that tend \nto have a lot of high-contrast edges.  The much harder problem of computing motion \nin low-contrast, high-noise outdoor environments still remains unsolved. \n\nWe  present two  motion algorithms.  Our first  algorithm is  a  \"delay-and-correlate\" \nscheme  operating on  spatial  edge  features  and  is  inspired  by  work  on  fly  vision \n(Hassenstein and Reichardt,  1956).  It yields a  non-monotonic response vs.  velocity \ncurve.  We  present  data from  a  chip  that implements it.  Our second algorithm is \na  \"facilitate-and-trigger\" scheme operating on  temporal edge features and yields a \nmonotonic response vs.  velocity curve.  Work is under way to implement our second \nalgorithm in  analog VLSI. \n\n2  The Delay-and-Correlate  Scheme \n\nConceptually,  there  are  two  stages  of  computation.  First,  the  zero-crossings  in \nthe  image  are  computed  and  then  the  motion  of these  zero-crossings  is  detected. \nThe  zero-crossing  circuitry  has  been  described  in \n(Bair  and  Koch,  1991).  We \nconcentrate on describing the motion circuitry. \n\nA schematized version of the chip is shown in Figure 1a.  Only four photo receptors in \nthe array are shown.  The 1-D image from the array of photoreceptors is filtered with \na spatial bandpass filter whose kernel is composed of a difference of two exponentials \n(implemented  with  resistive  grids).  The  outputs  of the  bandpass  filter  feed  into \nedge  detection  circuitry  that  output  a  bit  indicating  the  presence  or  absence  of \nan  edge  between  two  adjacent  pixels.  The  edges  on  the  chip  are  separated into \ntwo polarities, namely,  right-side-bright  (R) and left-side-brigbt  (L), which are kept \nseparate throughout  the  chip,  including  the  motion  circuitry.  For  comparison,  in \nbiology, edges are often separated into light-on edges and light-off edges.  The motion \ncircuits  are  sensitive  only  to  the  motion  of  those  edges  from  which  they  receive \ninputs.  They detect the motion of a  zero-crossing from  one location to an adjacent \nlocation  using a  Reichardt scheme as  shown  in  Figure  lb.  Each  motion  detecting \nunit  receives  two  zero-crossing inputs  ZCn  and  ZCn+21.  The ON-cells  detect  the \nonset of zero-crossings  (a rising  voltage  edge)  by  firing  a  pulse.  The units marked \nwith D's delay these pulses by an amount D,  controlled externally.  The correlation \nunits marked with X's logically AND a  delayed version of a  pulse from one location \nwith an undelayed  version of a  pulse  from  the adjacent location.  The output from \nthe left correlator is  sensitive to motion from  location n + 2 to location n  since the \nmotion delay is  compensated by the  built-in  circuit delay.  The outputs of the  two \ncorrelators are subtracted to yield the final motion signal. \nFigure 2a shows the \ncircuit details.  The boxes labelled with  pulse symbols represent axon circuits.  The \n\n1 ZCn+1  could have been used as well.  ZCn+2  was chosen due to wiring constraints and \n\nbecause it increases the baseline distance for  computing motion. \n\n\fVisual  Motion Computation in  Analog  VLSI  using Pulses \n\n783 \n\nA \n\nB \n\nPR3 \n\nBANDPASS  SPATIAL  FILTER \n\nM \n\nM \n\nMt~ \n\n+ \n\nFigure  l-(A)  The  bandpass  filtered  photoreceptor  signal  is  fed  to  the  edge \ndetectors  marked  with  E's.  The  motion  of  these  edges  is  detected  by  the \nmotion  detecting units  marked  with M's.  (B)  A  single motion detecting unit, \ncorresponding to a  \"M\"  unit  in fig.  A,  has a  Reichardt-like architecture. \n\naxon circuits generate a  single  pulse  of externally controlled width,  P, in  response \nto a  sharp  positive  transition in  their input,  but  remain  inactive  in  response  to a \nnegative  transition.  In  order  to  generate  pulses  that  are  delayed  from  the  onset \nof a  zero-crossing,  the output of one  axon  circuit,  with  pulse  width  parameter D, \nis  coupled  via an  inverter  to  the  input  of another axon  circuit,  with  pulse  width \nparameter P.  The multiplication operation is implemented by a simple logical AND. \nThe subtraction operation is implemented by  a  subtraction of two currents.  An off(cid:173)\nchip  sense  amplifier  converts  the  bidirectional  current  pulse  outputs of  the  local \nmotion detectors into active-low  or active-high  voltage  pulses.  The axon circuit  is \nshown in  Figure 2b.  Further details of its operation may  be found  in  (Sarpeshkar \net al.,  1992). \nFigure 3a shows how the velocity tuning curve is obtained.  If the image velocity,  v, \nis  positive,  and  ~x is  the distance  between  adjacent  zero-crossing locations,  then \nit  can  be  shown  that  the  output  pulse  width  for  the  positive-velocity  half of the \nmotion detector, tp,  is \n\ntp  =  u8(u), \n\n(1) \n\n\f784 \n\nSarpeshkar,  Bair,  and  Koch \n\nwhere \n\nu=P-I--DI, \n\nAx \nv \n\n(2) \n\nand  e(~) is  the  unit  step function.  If v  is  negative,  the same eqns.  apply  for  the \nnegative-velocity half of the motion detector except that the signs of Ax and tp are \nreversed. \n\nA \n\nI-----+C V REF \n\nFigure 2-(A) The circuitry implementing the Reichardt scheme of Figure lb, \nis  shown.  The  boxes  labelled  P  and  D  represent  axon  circuits  of pulse  width \nparameter  P  and  D,  respectively.  (B)  The  circuit  details  of an  axon  circuit \nthat  implements  an  ON-cell  of Figure  lb  are  shown.  The  input  and  output \nare  'Vin  and  Vout  respectively.  The circuit was  designed to  mimic the behavior \nof sodium and leak conductances at the node of Ranvier in an axon fiber.  The \npulse width of the output pulse, the refractory period following  its generation, \nand  the  threshold  height  of the  input  edge  needed  to  trigger  the  pulse  are \ndetermined  by the values of bias voltages  VD,  VR  and VL  respectively. \n\nExperimental Data \n\nFigure 4a shows  the outputs of motion detectors between zero-crossings 3 and 5,  7 \nand 9,  and  11  and  13,  denoted as  Mt3,  Mh,  and  Mtll, respectively.  For an edge \npassing from left to right, the outputs Mtll, Mt7 and Mt3  are excited in this order, \nand  they each  report a  positive velocity  (active  high  output that is  above  VREF)' \nFor an edge passing from right to left, the outputs Mt3, Mt7 and Mtll are excited in \nthis order and they each report a  negative velocity  (active low  output that is  below \nVREF)'  Note that the amplitudes of these pulses contain no speed information and \nonly signal the direction of motion.  Figure 4b shows that the output M t3  is  tuned \nto a  particular velocity.  As  the  rotational frequency  of a  cylinder with  a  painted \nedge  is  decreased from  a  velocity  corresponding to  a  motor voltage  of - 6.1 V  to a \nvelocity corresponding to a motor voltage of -1.3V, the output pulse width increases, \nthen decreases again, as the optimal velocity is traversed through.  A similar tuning \nIf the distance from the surface of \ncurve is observed for positive motor voltages. \nthe spinning cylinder to the center of the lens is  0,  the distance from  the center of \nthe lens  to the chip is  i,  the radius of the spinning cylinder is  R,  and its  frequency \n\n\fVisual  Motion Computation in Analog VLSI  using  Pulses \n\n785 \n\nP \n\n( \n\nD \n\nDelayed \nPulse \n\nTime \n\n(I) \n\nU ndelayed \n~  ~x \n.=:: \nPulse \n~~------~--~------~ \n\nv \n\nTime \n\nMotion Delay \n\nC~X) \n\n~ \n\n8 \n6 \n4 \n2 \n~  a \n~ -2 \n~ 5.  -4 \n~  -6 \no  -8 \n.-r.n \n\n\"0 \n~-10~-+--~--~-+--+-~ \nb.O \n\n-100 \nVeloci ty (degj sec) \n\n100 \n\na \n\nFigure 3-(a) The figure shows how the overlap between the delayed and unde(cid:173)\nlayed pulses gives rise to velocity tuning for  motion in the preferred direction. \n(b)  The figure  shows experimental data (circles) and a  theoretical fit  (line)  for \nthe  motion  unit  Mt3 's output response  vs.  angular velocity. \n\nof rotation is f, then the velocity,  v, of the moving edge as seen by  the chip is given \nby \n\nz \nv=27rfR-. \no \n\nThe angular velocity,  w,  of the moving edge,  is \n\nw = \n\n27rf R \n\no \n\n-\n\nv \n\"7. \nZ \n\n(3) \n\n(4) \n\nFigure 3b shows the output pulse width of M t3  plotted against the angular velocity \nof the edge  (w).  The data are fit  by a  curve  that com pu tes tp  vs.  w from  (1)- ( 4), \nusing measured values of ~x = 180 pm, 0  = 310 mm, i  = 17 mm, and R = 58 mm. \n\n\f786 \n\nSarpeshkar,  Bair,  and  Koch \n\n\u2022 \n\n, \n\nt \n\np \n\n\u2022 \n\n, .   \".n .. \n,n \n..  \u2022  II \n, \n\"u \n\nIt: ~ \n\n:, \n\n\"~ \n\nt \n\n,  II'  e ..... \n\nA \n\nt  \u2022 \n\n.. \n'  Mtll \n\u2022  \", . , .  I M t7 \n:, ;,!  Mta \nMta \n\n\u2022  .....  ;'  ;1 \n\n~.. \n: :.n:;, \n\nMt7 \n\nlJ~'\" , Itt  Mtll \n\n0.0  0.05  0.1  0.15 0.20 \n\nTime (Sec) \n\n.-0 \n\n,.--.... \n> \n\"\"0 \n\"-> \n\nlr.l \nN \n............, \n(l) \nb.O \n..... \nC1:'l \n.,;> \n~ \n\n.,;> \n;::j \n0.-\n.,;> \n::1 \n0 \n\nB \n+5.0 . \n.  1 \n111  . .  11. \n:iJ.,Q \n~ \n+1.5 \n\u00b11.a \n.. \n, \n-1.3 \n\nn \nI \nn \n\u2022 n \n\n\u2022 \n\njolll \n\n: \u2022 \n\n,I \n\n\u2022  U'  -1.5 t: \nI \n,  i  D \n\n-6.I \n\n0.0  1.0  2.0  3.0  4.0 \n\nTime (10-2  Sec) \n\nFigure 4-(A) The chip output waveforms  are active-high when the  motion is \nin  a  direction  such  that  motion  detectors  11,  7  and  3  are  stimulated  in  that \norder.  In the opposite direction  (3 - 7 - 11), the output waveforms are active(cid:173)\nlow.  (B)  The  figure  demonstrates  the  tuned  velocity  behavior  of the  motion \nunit Mt3  for various motor voltages  (in V).  Large positive voltages correspond \nto  fast  motion  in one  direction and  large  negative voltages correspond  to  fast \nmotion  in  the opposite direction. \n\n3  The Facilitate-and-Trigger  Scheme \n\nAlthough  the delay-and-correlate scheme works  well,  the output yields  ambiguous \nmotion  information  due  to  its  non-monotonic  velocity  dependence,  i.e.,  we  don't \nknow  if  the  output  is  small  because  the  velocity  is  too  slow  or  because  it is  too \nfast.  This problem may  be solved  by  aggregating the outputs of a  series of motion \ndetectors with overlapping tuning curves and progressively larger optimal velocities. \nThis solution is  plausible in physiology  but expensive in  VLSI.  We  were thus moti(cid:173)\nvated to create a  new  motion detector that possessed a  monotonic velocity  tuning \ncurve from  the outset. \n\nFigure  Sa  shows  the  architecture  of such  a  motion  detecting  unit:  The  need  for \ncomputing spatial edges is obviated by having a photoreceptor sensitive to temporal \nfeatures  caused  by  moving  edges,  i.e.,  sharp light  onsets/offsets (Delbriick,  1993). \nEach  ON-cell fires  a  pulse in  response to the onset of a  temporal edge.  The pulses \nare fed  to the facilitatory  (F)  and trigger (T) inputs of motion detectors tuned for \nmotion in  the left  or right  directions.  The facilitatory pulse defines a  time window \nof externally  controlled  width  F,  within  which  the  output  motion  pulse  may  be \nactivated by  the trigger pulse.  Thus, the rising edge of the trigger pulse must occur \nwithin  the  time  window  set  by  the  facilitatory  pulse  in  order  to  create a  motion \n\n\fVisual  Motion Computation in  Analog VLSI  using Pulses \n\n787 \n\noutput.  If this condition is  satisfied,  the output motion  pulse is  triggered  (begins) \nat  the  start of the  trigger  pulse  and  ends  at  the end  of the facilitatory  pulse;  its \nwidth, thus, encodes the arrival time difference between the onset pulses at adjacent \nlocations due to the motion delay.  Each  half of the motion detector only  responds \nto motion in the direction that corresponds to F  before  T.  Figure 5c shows that the \nvelocity tuning in this scheme is  monotonic.  It can be shown that the output pulse \nwidth  for  the positive half of the motion detector, tp ,  is  given  by \n\ntp=u8(u), \n\n(5) \n\nwhere u is  given  by, \n\n(6) \nThe dead zone  near the origin  may  be  made as  small  as  needed  by  increasing the \nFigure  5b  shows  a  compact  circuit  that \nwidth  of  the  facilitatory  pulse. \n\nu  =  F- - .  \n\nAx \nv \n\nPRn  A \n\nPRn+1 \n\nF \n\nQ) \n\n~ \n~ ....... \n~ \n\nQ) \n\n~ \n\n~ \n....... \n~ \n\nC \n\nFacilitation \nPulse \n\nTime \n\nTrigger \n\nPulse \n\nTime \n\n~ \n\n..c::: \nro\u00b7-\n~~ \n~Q) \nO~ \n;:1 \n~ \n\n~ 0..\"'0 \n\nF \n\nMotion Delay  (~X) \n\n~ \n\n~ \n\n~ \n\n..c::: \n0..\"'0 \nro\u00b7-\n~~ \n~ Q) \n0,.$ \n~ \n\n~F \n\nVelocity (v) \nFigure 5-(A) The motion detecting unit uses a  facilitate-and-trigger paradigm \ninstead  of a  delay-and-correlate paradigm as  a  basis  for  its operation.  (B)  A \ncompact circuit implements the motion detecting unit.  VT  is the trigger input, \nVF  is  the  facilitatory  input,  Vout  is  the  output and  VTH  is  a  bias  voltage.  (C) \nThe output response  has  a  monotonic  dependence on velocity. \n\nimplements the facilitate-and-trigger scheme.  The ON-cell,  subtraction and sense(cid:173)\namplifier circuitry are implemented as in  the Reichardt scheme. \n\n\f788 \n\nSarpeshkar,  Bair,  and Koch \n\nThe  facilitate-and-trigger  approach  requires  a  total  of 32  transistors  per  motion \nunit  compared  with  a  total  of 64  in  the  delay-and-correlate  approach,  needs  one \nparameter to  be controlled (F)  rather than two  (D  and P), and yields  monotonic \ntuning from  the outset.  We,  therefore, believe  that it will  prove  to be the superior \nof the two  approaches. \n\n4  Conclusions \n\nThe evaluations of onsets, delays and coincidences,  required for  computing motion \nare implemented very  naturally with  pulses rather than by  graded potentials as  in \nall other motion chips,  built so far.  Both of our motion algorithms time the motion \nof image features in  an efficient  fashion  by  using pulse computation. \n\nAcknowledgements \n\nMany thanks to Carver Mead for  his  encouragement, support and  use  of lab  facil(cid:173)\nities.  We  acknowledge  useful  discussions with  William  Bialek,  Nicola Franceschini \nand Tobias Delbriick.  This work was supported by  grants from  the Office  of Naval \nResearch  and  the California Competitive Technologies Program.  Chip fabrication \nwas  provided  by  MOSIS. \n\nReferences \n\nW.  Bair,  and  C.  Koch,  \"An  Analog  VLSI  Chip  for  Finding  Edges  from  Zero(cid:173)\ncrossings.\", In Advances in Neural  Information  Processing  Systems  Vol.  3,  R.  Lipp(cid:173)\nman,  J.  Moody,  D.  Touretzky,  eds.,  pp.  399-405,  Morgan  Kaufmann,  San Mateo, \nCA,199l. \n\nT.  Delbriick,  \"Investigations  of  Analog  VLSI  Visual  Transduction  and  Motion \nProcessing\",  PhD.  thesis,  Computation  and  Neural  Systems  Program,  Caltech, \nPasadena, CA,  1993. \nB.  Hassenstein  and  W.  Reichardt,  \"Systemtheoretische Analyse  der  Zeit,  Reihen(cid:173)\nfolgen,  und  Vorzeichenauswertung  bei  der  Bewegungsperzepion  des  Riisselkafers \nChlorophanus\",  Z.  Naturforsch.11b:  pp.  513-524,  1956. \n\nT. Horiuchi, W. Bair, A.  Moore, B. Bishofberger, J. Lazzaro, C. Koch,  \"Computing \nMotion  Using  Analog  VLSI  Vision  Chips:  an  Experimental  Comparison  among \nDifferent Approaches\", IntI.  Journal of Computer Vision,  8,  pp.  203-216,  1992. \n\nD.  Marr and  S.  Ullman,  \"Directional Selectivity  and its  Use  in  Early  Visual  Pro(cid:173)\ncessing\",  Proc.  R.  Soc.  Lond  B  211,  pp.  151-180,  1981. \n\nR.  Sarpeshkar,  L.  Watts,  C.  Mead,  \"Refractory  Neuron  Circuits\",  Internal  Lab \nMemorandum, Physics of Computation Laboratory, Pasadena, CA,  1992. \n\nJ. Tanner and C.  Mead,  \"An Integrated Optical Motion Sensor\",  VLSI Signal  Pro(cid:173)\ncessing  II,  S-Y  Kung,  R.E.  Owen,  and  J.G.  Nash,  eds.,  59-76,  IEEE  Press,  NY, \n1986. \n\n\f", "award": [], "sourceid": 625, "authors": [{"given_name": "Rahul", "family_name": "Sarpeshkar", "institution": null}, {"given_name": "Wyeth", "family_name": "Bair", "institution": null}, {"given_name": "Christof", "family_name": "Koch", "institution": null}]}