{"title": "Reading a Neural Code", "book": "Advances in Neural Information Processing Systems", "page_first": 36, "page_last": 43, "abstract": null, "full_text": "36 \n\nBialek, Rieke, van Steveninck and Warland \n\nReading  a  Neural  Code \n\nWilliam  Bialek,  Fred Rieke,  R.  R.  de  Ruyter van  Steveninck 1  and \n\nDavid Warland \n\nDepartment of Physics,  and \n\nDepartment of Molecular and  Cell Biology \n\nUniversity of California at Berkeley \n\nBerkeley,  California 94720 \n\nABSTRACT \n\nTraditional methods of studying neural coding characterize  the en(cid:173)\ncoding  of known  stimuli  in  average  neural  responses.  Organisms \nface  nearly the opposite task -\ndecoding short segments of a spike \ntrain to extract information about an unknown, time-varying stim(cid:173)\nulus.  Here  we  present strategies for  characterizing the  neural code \nfrom  the  point of view  of the  organism, culminating in  algorithms \nfor  real-time  stimulus reconstruction  based  on  a  single  sample  of \nthe spike train.  These methods are applied to the design and anal(cid:173)\nysis  of experiments on an  identified movement-sensitive neuron  in \nthe fly  visual system.  As far  as we  know this is  the first  instance in \nwhich a direct  \"reading\"  of the neural code has been  accomplished. \n\n1 \n\nIntroduction \n\nSensory systems receive information at extremely high rates, and much of this infor(cid:173)\nmation must  be  processed  in  real  time.  To understand  real-time  signal  processing \nin  biological systems we  must understand  the representation  of this  information in \nneural spike  trains.  \\Ve  ask  several  questions  in  particular: \n\n\u2022  Does  a  single  neuron  signal only  the  occurrence  of particular  stimulus  '\"fea(cid:173)\n\ntures,\"  or  can  the  spike  train represent a  continuous time-varying input? \n\n1 Rijksuniversiteit Groningen, Postbus 30.001,9700 RB  Groningen The  Netherlands \n\n\fReading a Neural Code \n\n37 \n\n\u2022  How much information is  carried by the spike train of a  single neuron? \n\u2022  Is  the  reliability  of the  encoded  signal limited  by noise  at the  sensory  input \n\nor by noise  and inefficiencies  in  the subsequent layers of neural  processing? \n\n\u2022  Is  the  neural  code  robust  to  errors  in  spike  timing,  or  do  realistic  levels  of \n\nsynaptic noise  place significant limits on  information transmission? \n\n\u2022  Do  simple analog computations on  the  encoded  signals correspond  to simple \n\nmanipulations of the spike  trains? \n\nAlthough neural coding has been studied for  more than fifty years, clear experimen(cid:173)\ntal answers to these questions have been elusive (Perkel &  Bullock,  1968; de  Ruyter \nvan Steveninck &  Bialek,  1988).  Here  we  present a new approach to the characteri(cid:173)\nzation of the neural code  which provides explicit and sometimes surprising answers \nto  these  questions  when  applied  to  an identified movement-sensitive neuron  in  the \nfly  visual system. \n\nWe approach the  study of spiking neurons from  the  point of view  of the organism, \nwhich, based only on the spike train, must estimate properties of an unknown time(cid:173)\nvarying stimulus.  Specifically we try to solve the problem of decoding the spike train \nto recover the stimulus in real time.  As far as we know our work is the first  instance \nin  which  it has  been  possible  to  \"read\"  the  neural code  in this  literal sense.  Once \nwe can  read  the code,  we  can address  the questions  posed  above.  In  this  paper  we \nfocus  on  the code  reading algorithm, briefly summarizing the results  which follow. \n\n2  Theoretical background \nThe  traditional approach  to  the  study of neural  coding characterizes  the  encoding \nprocess:  For  an  arbitrary stimulus  waveform s( r),  what  can  we  predict  about  the \nspike  train?  This  process  is  completely  specified  by  the  conditional  probability \ndistribution P[{tdls(r)] of the spike arrival times {til conditional on  the stimulus \ns( r).  In  practice  one  cannot  characterize  this  distribution  in  its  entirety;  most \nexperiments result in only  the lowest moment -\nthe  firing  rate as function of time \ngiven the stimulus. \n\nThe classic experiments of Adrian and others established that, for static stimuli, the \nresulting constant firing  rate provides a measure of stimulus strength.  This concept \nis  easily  extended  to  any  stimulus  waveform  which  is  characterized  by  constant \nparameters,  such  as  a  single frequency  or  fixed  amplitude sine  wave.  l\\'luch  of the \neffort in studying the encoding of sensory signals in the nervous system thus reduces \nto  probing  the  relation  between  these  stimulus parameters and  the  resulting firing \nrate.  Generalizations to time-varying firing  rates, especially in response  to periodic \nsignals,  have also  been  explored. \n\nThe  firing  rate  is  a  continuous  function  of  time  which  measures  the  probability \nper  unit  time  that  the  cell  will  generate  a  spike.  The  rate  is  thus  by  definition \nan  average  quantity;  it  is  not  a  property  of a  single  spike  train.  The  rate  can \nbe  estimated,  in  principle,  by  averaging over  a  large  ensemble  of redundant  cells, \n\n\f38 \n\nBialek, Rieke, van Steveninck and Warland \n\nor  by  averaging responses  of a  single  cell  over  repeated  presentations  of the  same \nstimulus.  This latter approach dominates the experimental study of spiking neurons. \nMeasurements of firing  rate  rely  on some  form of redundancy  -\neither  the  spatial \nredundancy  of identical cells  or  the temporal redundancy  of repeated  stimuli.  It is \nsimply not clear  that such redundancy exists in real sensory systems under  natural \nstimulus  conditions. \nIn  the  absence  of redundancy  a  characterization  of  neural \nresponses in terms of firing rate is oflittle relevance to the signal processing problems \nfaced  by  the  organism.  To say  that  \"information is  coded  in firing  rates\"  is  of no \nuse  unless  one  can explain  how  the  organism  could  estimate  these  firing  rates  by \nobserving  the spike trains of its own neurons. \n\nWe believe that none of the existing approaches2  to neural coding addresses the basic \nproblem of real-time signal processing with neural spike trains:  The organism must \nextract information about continuously varying stimulus waveforms using only the \ndiscrete  sequences  of spikes.  Real-time  signal  processing  with  neural  spike trains \nthus involves some sort of interpolation between the spikes  that allows the organism \nto estimate a  continuous function  of time. \n\nThe most basic problem of real-time signal processing is to decode the spike train and \nrecover  an  estimate  of the  stimulus  waveform  itself.  Clearly  if we  can  accomplish \nthis  task  then  we  can begin  to understand  how spike  trains can  be  manipulated to \nperform  more  complex  computations;  we  can  also  address  the  quantitative  issues \noutlined  in  the  Introduction.  Because  of the  need  to  interpolate  between  spikes, \nsuch decoding is not a simple matter of inverting the conventional stimulus-response \n(rate) relations.  In fact it is  not obvious a priori that true decoding is even possible. \n\nOne  approach  to  the  decoding  problem  is  to  construct  models  of  the  encoding \nprocess,  and  proceed  analytically  to  develop  algorithms  for  decoding  within  the \ncontext of the  model  (Bialek  &  Zee,  1990).  Using  the  results  of this  approach  we \ncan predict that linear filtering will,  under some conditions, be an effective decoding \nalgorithm, and we  can determine  the form of the filter  itself.  In this paper we  have \na  more  limited  goal,  namely  to  see  if the  class  of decoding  algorithms  identified \nby  Bialek  and  Zee  is  applicable  to  a  real  neuron.  To  this  end  we  will  treat  the \nstructure  of the  decoding filter  as  unknown, and find  the  \"best\"  filter  under  given \nexperimental conditions. \nWe  imagine  building a  set  of (generally  non-linear)  filters  {Fn}  which  operate  on \nthe  spike train to produce  an estimate of the stimulus.  If the spikes arrive at times \n{td, we  write our estimate of the  signal as  a  generalized  convolution, \n\ni \n\ni,j \n\n(1) \n\n2Higher moments of the  conditional  probability  P[{t i}ls(r)],  such as  the  inter-spike interval \ndistribution  (Perkel  &  Bullock,  1968)  are  also average properties,  not  properties of single spike \ntrains, and hence may not be relevant to real-time signal processing.  White-noise methods  (Mar(cid:173)\nmarelis &  Marmarelis, 1978) result in models which predict the time-varying firing rate in response \nto arbitrary input waveforms and thus suffer the same limitations as other rate-based approaches. \n\n\fReading a Neural Code \n\n39 \n\nHow  good are  the  reconstructions?  We separate systematic  and random errors  by \nintroducing a  frequency  dependent  gain  g(w)  such  that  (ls(w)l) = g(w) (lsut(w)l). \nThe resulting gain is approximately unity through a reasonable bandwidth.  Further, \nthe  distribution  of deviations  between  the  stimulus and  reconstruction  is  approx(cid:173)\nirr.ately  Gaussian.  The  absence  of systematic  errors  suggests  that  non-linearities \nin  the  reconstruction  filter  are  unlikely  to  help.  Indeed,  the  contribution from  the \nst.. ~ond order  term in  Eq.  (1)  to the  reconstructions  is  negligible. \n\no  ~---------------------------, \n\n-~ --\"-\n\":' --;... \n\n'\" \n~ \"0 \n... \n\"in \n:3 \n~  ~ \n.::; \nu \n~ \n:ii' \n\n~ \n.~  ~ \n~ \n\n~  ~----~ ____ ~ __ ~ ____ ~ ____ -J \n\no \n\n10 \n\nfrequency (Hz) \n\nFigure  2:  Spectral density  of displacement  noise  from  our  reconstruction  (upper \ncurve).  By multiplying the displacement noise  level by a  bandwidth, we  obtain the \nsquare  of the  angular  resolution  of HI  for  a  step  displacement.  For  a  reasonable \nbandwidth  the  resolution  is  much  less  than  the  photoreceptor  spacing,  1.350 \n\"hyperacuity.\"  Also shown is  the  limit to  the resolution of small displacements set \nby noise in the photoreceptor array (lower curve). \n\n-\n\nWe  identify  the  noise  at  frequency  w  as  the  difference  between  the  stimulus and \nthe  normalized  reconstruction,  n(w)  =  s(w)  - g(w )Sed (w).  \\Ve  then  compute  the \ndpectral density (noise power per unit bandwidth) of the displacement noise (Fig 2). \nThe noise level achieved in HI is  astonishing; with a  one second integration time an \nobserver of the spike train in HI could judge the amplitude of a low frequency dither \nto 0.01\u00b0 - more than one hundred times less than the photoreceptor spacing!  If the \nfiY'f  neural circuitry is noiseless,  the fundamental limits to displacement  resolution \n\n\f40 \n\nBialek, Rieke, van Steveninck and Warland \n\nstimulus, \n\nFl(T)  =  _e- 1wr \n\nJ dw \n\n27r \n\n. \n\n(s(w) Lj e- iwtj ) \n(Li,j eiW(t.-t j \u00bb) \n\n\u2022 \n\n(2) \n\nThe averages ( .. . ) are  with respect  to an ensemble of stimuli S ( T). \n\n2.  Minimize  X2  with  respect  to  purely causal functions.  This  may be  done  an(cid:173)\nalytically,  or  numerically  by expanding  F 1 ( T)  in  a  complete  set  of functions \nwhich vanish at negative times, then minimizing X2  by varying the coefficients \nof the  expansion.  In  this  method  we  must  explicitly introduce a  delay  time \nwhich  measures  the lag between the  true stimulus and our reconstruction. \n\nWe  use  the filter  generated  from the first  method  (which is  the  best  possible linear \nfilter)  to check  the filter  generated  by the second  method.  Fig.  1 illustrates recon(cid:173)\nstructions  using  these  two methods.  The  filters  themselves  are  also  shown  in  the \nfigure;  we  see  that both methods give essentially the  same  answer. \n\n~  ~----------------------------~ \n\n... \no  ~----------------------------~ \n\n~ \n\n~ \n\n'-' \n::I \n'\"  0 \n\n-\n_N \n-\n\n~ \n~ \n6b \n~o  I \n, \n, \n>. \ni\" \n\\I \n'u  0 \n0'1' \n~ \n1:) \n> \n\n~ \n\nq \n\n~ \n,~ \nI \n~I  ,I \n, \n\\' \n.  \" \nI. \n, \n, \nI , \nI  I \n, \nrf  I \nI \n' \\  \nI \nI \\ !  \\ \n' \nIf \n\n, \n\" \nv \n\nI \nI \n\\ \n\nI \n\nI \n\n\u2022 \n1\\ \n\" \nP \ni \nI~, \n\nd~ \n\\ \n\nJ \n\nrJ~ \n\n1\\' \n/ . \n\nI \nI' \n~ I' \n1\\ \n\\ \n'I  I \n11/ '  \nIv \n, \n, \n\\ ! \nV \n\nV \n\n\" \n\n:;: \nS' \n:.; \nr. \n<Il \n~ \n::I \nt,~ \n::I \n~ \n\n-\n-\nV~ \n, . \n~ \n... \n.  c,.o \n:1 \nII \n,.. \n~ \n... \nv \n.;:: \nto: \n\ni \n\nI \n\n~ \n\n~ \n\niii  Ull \n\nI \n\n'H  II  III I \n\nI \n\n'II I  nil \n\nI \n\nI  II \n\n, \n\nII \n\n~ \n\n2000 \n\n2100 \n\n2300 \n2200 \ntime  (msec) \n\n0 \n'1' \n\n2400 \n\n2500 \n\n\u00b750 \n\no \n\n50 \n\n100 \n\n150 \n\ntime  (msec) \n\nFigure  1:  First  order  reconstruction  se,,( T)  using  method  1  (solid  line).  The \nst.imulus  is  shown here  as a  dotted  line  for  comparison.  The reconstruction  shown \nis for  a segment of the spike train which was not used in the filter  calculations.  The \nspike train is shown at the bottom of the figure,  where  the negative spikes are from \nthe  \"other  eye\"  (cf.  footnote  3).  Both  stimulus and  reconstruction  are  smoothed \nwith a 5 msec half-wid th Gaussian filter.  The filters calculating using both methods \nare shown on  the right. \n\n\fReading a Neural Code \n\n41 \n\nWe  define  the  optimal filter  to  be  that  which  minimizes  X2  = f dtls(t) - sest(t)12 , \nwhere  s(t)  is  the  true  stimulus,  and  the  integration  is  over  the  duration  of  the \nexperiment. \n\nTo insure  that the filters  we calculate allow real-time decoding, we  require  that the \nfilters  be  causal,  for  example  FI(T  <  0)  =  O.  But  the  occurrence  of a  spike  at \nt'  conveys  information about  the  stimulus at a  time  t  < t',  so  we  must  delay  our \nestimate  of the  stimulus  by  some  time  Tdelay  >  t'  - t.  In  general  we  gain  more \ninformation  by  increasing  the  delay,  so  we  face  a  tradeoff:  Longer  waiting  times \nallow us  to gain more information but introduce longer reaction times to important \nstimuli.  This  tradeoff is  exactly  the  tradeoff faced  by  the  organism in  reacting  to \nexternal stimuli based on  noisy and incomplete information. \n\n3  Movelnent detection in  the blowfly visual system \n\nWe  apply  our  methods  in  experiments  on  a  single  wide  field,  movement-sensitive \nneuron  (H 1)  in  the  visual  system  of the  blowfly  Calliphora  erythrocephela.  Flies \nand  other  insects  exhibit  visually  guided  flight;  during  chasing  behavior  course \ncorrections can occur on time scales as short as 30 msec  (Land &  Collett,  1974).  H1 \nappears  to be an obligatory link in  this control loop, encoding wide field  horizontal \nmovements  (Hausen,  1984).  Given  that  the  maximum  firing  rate  in  H1  is  100-\n200 Hz,  behavioral decisions must be based on the information carried by just a  few \nspikes  from  this  neuron.  Further,  the  horizontal motion  detection  system consists \nof only  a  handful  of neurons,  so  the  fly  has  no  opportunity  to  compute  average \nresponses  (or  firing  rates). \n\nIn the experiments described here,  the fly  is looking at a rigidly moving random pat(cid:173)\ntern  (de Ruyter van Steveninck, 1986).  The pattern is presented on an oscilloscope, \nand  moved horizontally every  500  J-lsec  in  discrete  steps  chosen  from  an ensemble \nwhich approximates Gaussian white noise.  This time scale is short enough  that we \ncan  consider  the  resulting  stimulus waveform s(t)  to  be the  instantaneous angular \nvelocity.  We record  the spike arrival times {til extracellularly from the H1  neuron.3 \n\n4  First order reconstructions \nTo  reconstruct  the  stimulus  waveform  requires  that  we  find  the  filter  FI  which \nminimizes X2.  We  do this  in two different  ways: \n\n1.  Disregard  the  constraint  that  the  filter  be  causal.  In  this  case  we  can  write \nan explicit formula for  the optimal filter  in terms  of the  spike  trains and  the \n\n3 There is one further caveat to the experiment.  The firing rate in HI  is increased for  back-to(cid:173)\n\nfront motion and is decreased for front-to-back motion;  the dynamic range is  much greater in the \nexcitatory direction.  The fiy,  however,  achieves high sensitivity in both directions by combining \ninformation from both eyes.  Because front-to-back motion in one eye corresponds to back-to-front \nmotion in  the other eye,  we  can simulate the two eye case while recording from only  one HI  cell \nby using  an antisymmetric stimulus  waveform.  We combine  the information coded in  the spike \ntrains  corresponding  to  the  two  \"polarities\"  of the stimulus  to  obtain  the information available \nfrom both HI  neurons. \n\n\f42 \n\nBialek, Rieke, van Steveninck and Warland \n\nare set  by noise  in  the  photoreceptor  array.  We have calculated  these  limits in  the \ncase  where  the  displacements  are  small,  which  is  true  in  our  experiments  at  high \nfrequencies.  In comparing these  limits with  the results  in  HI  it is  crucial  that  the \nphotoreceptor signal and noise characteristics (de Ruyter van Steveninck,  1986) are \nmeasured  under  the  same  conditions  as  the  HI  experiments  analyzed  here.  It is \nclear from  Fig.  2  that  HI approaches  the  theoretical  limit  to its  performance.  We \nemphasize that the noise spectrum in Fig. 2 is  not a  hypothetical measure of neural \nperformance.  Rather  it  is  the  real  noise  level  achieved  in  our  reconstructions.  As \nfar  as  we  know this is  the first  instance in which the  equivalent spectral noise  level \nof a  spiking neuron  has  been  measured. \n\nTo  explore  the  tradeoff  between  the  quality  and  delay  of  the  reconstruction  we \nmeasure  the  cross-correlation  of  the  smoothed  stimulus  with  the  reconstructions \ncalculated  using  method  2  above  for  delays of 10-70  msec.  For a  delay of 10  msec \nthe  reconstruction  carries  essentially  no  information;  this  is  expected  since  a  de(cid:173)\nlay of 10  msec  is  close  to  the  intrinsic delay for  phototransduction.  As  the  delay \nis  increased  the  reconstructions  improve,  and  this  improvement  saturates  for  de(cid:173)\nlays greater  than 40  msec,  close  to  the  behavioral reaction  time  of 30  msec  -\nthe \nstructure  of  the  code  is  well  matched  to  the  behavioral  decision  task  facing  the \norganism. \n\n5  Conclusions \nLearning  how  to  read  the  neural  code  has  allowed  us  to  quantify  the  information \ncarried  in  the  spike  train  independent  of assumptions  regarding  the  structure  of \nthe  code.  In  addition, our analysis  gives some  hopefully more general insights into \nneural coding and computation: \n\n1.  The  continuously varying movement signal encoded in the firing  of H1  can  be  re(cid:173)\nconstructed by  an  astonishingly  simple linear filter.  If neurons summed  their inputs \nand  marked  the  crossing  of thresholds  (as  in  many  popular  models),  such  recon(cid:173)\nstructions  would  be  impossible;  the  threshold  crossings  are  massively  ambiguous \nindicators  of the signal  waveform.  We  have  carried  out  similar  studies  on  a  stan(cid:173)\ndard model neuron (the FitzHugh-Nagumo model), and find  results similar to those \nin  the  HI  experiments.  From the  model  neuron  studies  it  appears  that  the  linear \nrepresentation of signals in spike trains is  a  general property of neurons, at least in \na  limited  regime  of their  dynamics.  In  the  near future  we  hope  to investigate this \nstatement in other sensory systems. \n2.  The  reconstruction  is  dominated  by  a  \"window\"  of - 4 0 msec  during  which \nat  most  a  few  spikes  are  fired.  Because  so  few  spikes  are  important,  it  does  not \nmake  sense  to  talk  about  the  \"firing  rate\"  -\ntime  from \nobservations of the spike  train is at least  as hard as estimating the stimulus itself! \n\nestimating  the  rate  vs. \n\n3.  The  quality of the reconstructions can  be  improved by  accepting longer delays,  but \nthis  improvement  saturates  at  - 30 - 40 msec,  in  good  agreement  with  behavioral \ndecision  times. \n\n\fReading a Neural Code \n\n43 \n\n4.  Having  decoded  the  neural signal we  obtain  a  meaningful estimate  of the  noise \nlevel in the system and the  information content of the  code.  H1  accomplishes a real(cid:173)\ntime  version of hyper acuity, corresponding to a  noise  level near  the limits imposed \nby the quality of the sensory input.  It appears that this system is close to achieving \noptimal real-time signal processing. \n\n5.  From  measurements of the  fault  tolerance  of the  code  we  can  place  requirements \non  the  noise levels in  neural circuits using  the  information  coded in H1.  One of the \nstandard  objections  to  discussions  of  \"spike  timing\"  as  a  mechanism of coding  is \nthat  there  are  no  biologically  plausible  mechanisms  which  can  make  precise  mea(cid:173)\nsurements  of spike  arrival  times.  We have  tested  the  required  timing precision  by \nintroducing timing errors into the spike train and characterizing the resulting recon(cid:173)\nstructions.  Remarkably  the code  is  \"fault  tolerant,\"  the  reconstructions  degrading \nonly slightly when  we  add  timing errors of several msec. \n\nFinally,  we  wish  to emphasize our own surprise  that it is  so  simple  to recover  time \ndependent signals from neural spike trains.  The filters  we  have constructed  are  not \nvery complicated, and they are linear.  These results suggest that the representation \nof time-dependent sensory data in the nervous system is  much simpler than we migh t \nhave expected.  We  suggest  that, correspondingly, simpler models of sensory signal \nprocessing  may be  appropriate. \n\n6  Acknowledglnents \n\nWe thank  W. J.  Bruno,  M.  Crair,  L.  Kruglyak,  J.  P.  Miller,  W.  G.  Owen,  A.  Zee, \nand  G.  Zweig  for  many  helpful discussions.  This  work  was supported  by  the  Na(cid:173)\ntional Science Foundation through a Presidential Young Investigator Award to WB, \nsupplemented by funds from  Cray Research  and Sun Microsystems,  and  through  a \nGraduate Fellowship to FR. DW was supported in part by the System'S  and Integra(cid:173)\ntive Biology Training Program of the National Institutes of Health.  Initial work was \nsupported  by  the  Netherlands Organization for  Pure Scientific  Research  (ZWO). \n\n7  References \nW.  Bialek  and  A.  Zee.  J.  Stat.  Phys.,  in press,  1990. \nK.  Hausen.  In M.  Ali,  editor,  Photoreception  and  Vision  in  Invertebrates.  Plenum \nPress,  New  York  and  London,  1984. \nM.  Land and T.  Collett.  J.  Compo  Physiol.,  89:331,  1974. \nP.  Marmarelis and  V.  Marmarelis.  Analysis of Physiological Systems.  The  White \nNoise  Approach.  Plenum Press,  New  York,  1978. \nD.  Perkel and T.  Bullock.  Neurosciences.  Res.  Prog.  Bull., 6:221,  1968. \nR.  R.  de  Ruyter  van Steveninck and  W.  Bialek.  Proc.  R.  Soc.  Lond.  B,  234:379, \n1988. \nR.  R.  de  Ruyter  van Steveninck.  Real-time  Performance  of a  Movement-sensitive \nNeuron  in  the  Blowfly  Visual  System.  Rijksuniversiteit  Groningen,  Groningen, \nNetherlands,  1986. \n\n\f", "award": [], "sourceid": 272, "authors": [{"given_name": "William", "family_name": "Bialek", "institution": null}, {"given_name": "Fred", "family_name": "Rieke", "institution": null}, {"given_name": "Robert", "family_name": "van Steveninck", "institution": null}, {"given_name": "David", "family_name": "Warland", "institution": null}]}