Part of Advances in Neural Information Processing Systems 10 (NIPS 1997)
John J. Hopfield, Carlos Brody, Sam Roweis
Most computational engineering based loosely on biology uses contin(cid:173) uous variables to represent neural activity. Yet most neurons communi(cid:173) cate with action potentials. The engineering view is equivalent to using a rate-code for representing information and for computing. An increas(cid:173) ing number of examples are being discovered in which biology may not be using rate codes. Information can be represented using the timing of action potentials, and efficiently computed with in this representation. The "analog match" problem of odour identification is a simple problem which can be efficiently solved using action potential timing and an un(cid:173) derlying rhythm. By using adapting units to effect a fundamental change of representation of a problem, we map the recognition of words (hav(cid:173) ing uniform time-warp) in connected speech into the same analog match problem. We describe the architecture and preliminary results of such a recognition system. Using the fast events of biology in conjunction with an underlying rhythm is one way to overcome the limits of an event(cid:173) driven view of computation. When the intrinsic hardware is much faster than the time scale of change of inputs, this approach can greatly increase the effective computation per unit time on a given quantity of hardware.
1 Spike timing Most neurons communicate using action potentials - stereotyped pulses of activity that are propagated along axons without change of shape over long distances by active regenerative processes. They provide a pulse-coded way of sending information. Individual action potentials last about 2 ros. Typical active nerve cells generate 5-100 action potentials/sec.
Most biologically inspired engineering of neural networks represent the activity of a nerve cell by a continuous variable which can be interpreted as the short-time average rate of generating action potentials. Most traditional discussions by neurobiologists concerning how information is represented and processed in the brain have similarly relied on using "short term mean firing rate" as the carrier of information and the basis for computation. But this is often an ineffective way to compute and represent information in neurobiology.
*Dept. of Molecular Biology, Princeton University. jhopfield@watson.princeton. edu
t Computation & Neural Systems, California Institute of Technology.
Computing with Action Potentials
167
To define "short term mean firing rate" with reasonable accuracy, it is necessary to either wait for several action potentials to arrive from a single neuron, or to average over many roughly equivalent cells. One of these necessitates slow processing; the other requires redundant "wetware".
Since action potentials are short events with sharp rise times, action potential timing is another way that information can be represented and computed with ([Hopfield, 1995]). Action potential timing seems to be the basis for some neural computations, such as the determination of a sharp response time to an ultrasonic pulse generated by the moustache bat. In this system, the bat generates a 10 ms pulse during which the frequency changes monotonically with time (a "chirp"). In the cochlea and cochlear nucleus, cells which are responsive to different frequencies will be sequentially driven, each producing zero or one action potentials during the time when the frequency is in their responsive band. These action potentials converge onto a target cell. However, while the times of initiation of the action potentials from the different frequency bands are different, the length and propagation speed of the various axons have been coordinated to result in all the action potentials arriving at the target cell at the same time, thus recognizing the "chirped" pulse as a whole, while discriminating against random sounds of the same overall duration. Taking this hint from biology, we next investigate the use of action potential timing to rep(cid:173) resent information and compute with in one of the fundamental computational problems relevant to olfaction, noting why the elementary "neural net" engineering solution is poor, and showing why computing with action potentials lacks the deficiencies of the conven(cid:173) tional elementary solution.
2 Analog match The simplest computational problem of odors is merely to identify a known odor when a single odor dominates the olfactory scene. Most natural odors consist of mixtures of sev(cid:173) eral molecular species. At some particular strength a complex odor b can be described by the concentrations Nt of its constitutive molecular of species i. If the stimulus intensity changes, each component increases (or decreases) by the same multiplicative factor. It is convenient to describe the stimulus as a product of two factors, an intensity .A and normal(cid:173) ized components n~ as: