{"title": "Neural Implementation of Motivated Behavior: Feeding in an Artificial Insect", "book": "Advances in Neural Information Processing Systems", "page_first": 44, "page_last": 51, "abstract": null, "full_text": "44 \n\nBeer and Chiel \n\nNeural  Implementation  of Motivated  Behavior: \n\nFeeding  in  an  Artificial Insect \n\nRandall D.  Beerl,2 and Hillel J.  Chiel2 \n\nDepartments of 1 Computer Engineering and Science,  and  2Biology \nand  the Center for  Automation and Intelligent Systems Research \n\nCase Western Reserve  University \n\nCleveland,  OH  44106 \n\nABSTRACT \n\nMost  complex  behaviors  appear  to be governed  by  internal  moti(cid:173)\nvational  states or  drives  that  modify  an  animal's  responses  to  its \nenvironment.  It is  therefore of considerable  interest to understand \nthe  neural basis of these  motivational states.  Drawing upon work \non  the  neural  basis  of feeding  in  the  marine  mollusc  Aplysia,  we \nhave  developed  a  heterogeneous  artificial  neural  network  for  con(cid:173)\ntrolling the feeding behavior of a simulated insect.  We demonstrate \nthat feeding in this artificial insect shares many characteristics with \nthe motivated behavior of natural animals. \n\nINTRODUCTION \n\n1 \nWhile an animal's external environment certainly plays an extremely important role \nin  shaping its  actions,  the  behavior  of even  simpler  animals  is  by  no  means solely \nreactive.  The response of an animal to food, for  example, cannot be explained only \nin terms of the physical stimuli involved.  On two different  occasions, the very same \nanimal  may  behave  in  completely  different  ways  when  presented  with  seemingly \nidentical  pieces  of food  (e.g.  hungrily  consuming  it  in  one  case  and  ignoring  or \neven  avoiding it in another).  To account for  these  differences,  behavioral scientists \nhypothesize  internal motivational states or  drives  which  modulate an  animal's re(cid:173)\nsponse to its environment.  These internal factors playa particularly important role \nin  complex  behavior,  but  are  present  to  some  degree  in  nearly  all  animal  behav(cid:173)\nior.  Behaviors which exhibit an extensive dependence on motivational variables are \ntermed  motivated behaviors. \n\n\fNeural Implementation or Motivated Behavior:  Feeding in an Artificial Insect \n\n45 \n\nWhile a rigorous definition is difficult to state, behaviors spoken of as motivated gen(cid:173)\nerally exhibit some subset of the following six characteristics  (Kupfermann,  1974): \n(1) grouping and sequencing of component behaviors in time, (2) goal-directedness: \nthe sequence of component behaviors generated can often be understood only by ref(cid:173)\nerence to some internal goal, (3)  spontaneity:  the behavior can occur in the absence \nof any  recognizable  eliciting stimuli,  (4)  changes  in  responsiveness:  the  effect  of a \nmotivational state  varies  depending  upon  an  animal's  level  of arousal,  (5)  persis(cid:173)\ntence:  the behavior  can greatly outlast any  initiating stimulus,  and (6)  associative \nlearning. \n\nMotivational  states  are  pervasive  in  mammalian  behavior.  However,  they  have \nalso  proven  to  be essential  for  explaining the  behavior  of simpler  animals  as  well. \nUnfortunately, the explanatory utility of these internal factors is limited by the fact \nthat they are hypothetical constructs, inferred by  the theorist to intervene between \nstimulus and action in order to account  for  otherwise inexplicable responses.  What \nmight be the  neural basis of these motivational states? \n\nIn  order  to  explore  this  question,  we  have  drawn  upon  work  on  the  neural  basis \nof feeding  in  the  marine  mollusc  Aplysia  to  implement  feeding  in  a  simulated  in(cid:173)\nsect.  Feeding is  a  prototypical motivated behavior  in which  attainment of the goal \nobject  (food)  is  clearly  crucial  to  an  animal's  survival.  In  this  case,  the  relevant \nmotivational state is  hunger.  When an  animal is  hungry,  it will  exhibit a  sequence \nof  appetitive  behaviors  which  serve  to  identify  and  properly  orient  the  animal  to \nfood.  Once  food  is  available,  consummatory  behaviors  are  generated  to  ingest  it. \nOn  the  other  hand,  a  satiated  animal  may  ignore  or  even  avoid  sensory  stimuli \nwhich suggest the presence of food  (Kupfermann,  1974). \n\nThis effort  is  part of a  larger  project aimed  at designing  artificial nervous  systems \nfor  the  flexible  control  of complete  autonomous  agents  (Beer,  1989).  In  addition \nto feeding,  this artificial insect is currently capable of locomotion (Beer,  Chiel, and \nSterling, 1989; Chiel and Beer,  1989), wandering,  and edge-following, and possesses \na  simple  behavioral  hierarchy  as  well.  A  central  theme of this  work  has  been  the \nutilization  of biologically-inspired  architectures  in our  neural  network  designs.  To \nsupport  this  capability, we  make use  of model neurons  which  capture some of the \nintrinsic properties of nerve  cells. \n\nThe  simulated  insect  and  the  environment  in  which  it  exists  is  designed  as  fol(cid:173)\nlows.  The  insect  has six  legs,  and  is  capable  of statically  stable  locomotion  and \nturning.  Its head  contains  a  mouth  which  can open  and  close,  and  its mouth  and \ntwo antennae possess  tactile and chemical sensors.  The insect possesses  an internal \nenergy  supply  which  is  depleted  at  a  fixed  rate.  The  simulated environment  also \ncontains unmovable obstacles and circular food  patches.  The food  patches emit an \nodor whose  intensity is  proportional  to the size  of the patch.  As  this odor  diffuses \nthrough the environment, its intensity falls  off as  the inverse square of the distance \nfrom  the center  of the patch.  Whenever  the insect's mouth  closes  over  a  patch  of \nfood,  a  fixed  amount of energy is  transferred from the patch  to the insect. \n\n\f46 \n\nBeer and Chiel \n\nAnlenna Chemical Sensor \n\nAnlenna Chemical Sensor \n\nleft Turn \n\nRighi Turn \n\nFeeding Arousal \n\nEnergy Sensor \n\nFigure 1:  Appetitive Controller \n\n2  APPETITIVE COMPONENT \nThe appetitive component of feeding is responsible for  getting a hungry  insect to a \nfood patch.  To accomplish this task, it utilizes the locomotion, wandering, and edge(cid:173)\nfollowing  capabilities  of the insect.  The  interactions  between  the  neural  circuitry \nunderlying  these  behaviors  and  the  feeding  controller  presented  in  this  paper  are \ndescribed  elsewhere (Beer,  1989).  Assuming that the insect is  already close enough \nto a food  patch that the chemical sensors in its antennae can detect an odor signal, \nthere are two separate issues which must be addressed by this phase of the behavior. \nFirst, the insect must use the information from the chemical sensors in its antennae \nto turn itself toward the food patch as it walks.  Second, this orientation should only \noccur when  the insect is  actually in need of energy.  Correspondingly, the appetitive \nneural controller  (Figure  1)  consists of two distinct components. \nThe orientation component is comprised of the upper six  neurons in  Figure  1.  The \nodor signals detected by the chemical sensors in each antenna (ACS)  are compared \n(by  LOS  and  ROS),  and  the  difference  between  them is  used  to  generate  a  turn \ntoward the stronger side by exciting the corresponding turn interneuron (LT or RT) \nby  an  amount  proportional  to  the size  of the  difference.  These  turn  interneurons \nconnect  to the motor  neurons controlling the lateral extension of each front  leg. \nThe second  component  is responsible  for  controlling whether or  not  the insect  ac-\n\n\fNeural Implementation of Motivated Behavior:  Feeding in an Artificial Insect \n\n47 \n\ntually  orients  to  a  nearby  patch of food.  This  decision  depends  upon  its internal \nenergy level, and is controlled by the bottom three neurons in Figure 1.  Though the \nodor gradient is continuously being sensed, the connections to the turn interneurons \nare  normally  disabled,  preventing  access  of this  information  to the  motor  appara(cid:173)\ntus  which  turns the insect.  As  the insect's energy  level  falls,  however,  so  does  the \nactivity  of its  energy  sensor  (ES).  This  decreasing  activity  gradually  releases  the \nspontaneously  active  feeding  arousal  neuron  (FA)  from  inhibition.  When  activity \nin FA  becomes sufficient  to fire  the search  command  neuron  (SC),  the connections \nbetween  the  odor  strength  neurons  and  the  turn  neurons  are  enabled  by  gating \nconnections from SC,  and the insect begins to orient to food. \n\n3  CONSUMMATORY  COMPONENT \nOnce the appetitive controller has successfully oriented the insect to food,  the con(cid:173)\nsummatory component of the behavior is triggered.  This phase consists of rhythmic \nbiting movements which persist until sufficient food has been ingested.  Like the ap(cid:173)\npetitive  phase,  consummatory behavior  should only  be  released  when  the insect  is \nin need of energy.  In  addition,  an animal's interest in feeding  (its feeding  arousa~, \nmay be a function of more than just its energy requirements.  Other factors, such as \nthe exposure of an animal to the taste, odor, or tactile sensations of food,  can signif(cid:173)\nicantly increase  its feeding  arousal.  This relationship  between feeding  and arousal, \nin  which  the  very  act  of feeding  further  enhances  an  animal's  interest  in  feeding, \nleads to a  form of behavioral hysteresis.  Once food  is encountered,  an animal may \nfeed  well  beyond  the internal energy requirements which initiated  the behavior.  In \nmany animals,  this hysteresis is  thought to playa role in the patterning of feeding \nbehavior  into discrete  meals  rather than continuous grazing (Susswein,  Weiss,  and \nKupfermann,  1978).  At  some  point,  of course,  the  ingested  food  must  be  capable \nof overriding  the  arousing  effects  of consummatory  behavior,  or  the  animal  would \nnever  cease to feed. \n\nThe  neural  controller  for  the  consummatory  phase  of feeding  is  shown  in  Figure \n2.  When  chemical  (MCS)  and  tactile  (MTS)  sensors  in  the  mouth  signal  that \nfood  is  present  (FP),  and  the  insect  is  sufficiently  aroused  to  feeding  (FA),  the \nconsummatory command neuron (CC) fires.  The conjunction of tactile and chemical \nsignals is required in order to prevent attempts to ingest  nonfood patches and, due \nto  the  diffusion  of odors,  to  prevent  biting  from  beginning  before  the food  patch \nis  actually  reached.  Once  CC  fires,  it  triggers  the  bite  pacemaker  neuron  (BP) \nto generate rhythmic  bursts which  cause  a  motor neuron  (MO)  to open  and  close \nthe mouth.  Because  the threshold of the  consummatory command neuron (CC)  is \nsomewhat  lower  than  that  of the  search  command  neuron  (SC),  an  insect  which \nis  not sufficiently  aroused  to orient to food  may nevertheless  consume food  that is \ndirectly  presented  to its mouth. \n\nThe motor neuron controlling the mouth also makes an excitatory connection onto \nthe  feeding  arousal  neuron  (FA),  which  in  turn  makes  an  excitatory  modulatory \nsynapse  onto  the  connection  between  the  consummatory  command  neuron  (CC) \n\n\f48 \n\nBeer and Chiel \n\nMouth Tact~e Sensor \n\nMouth Chemical Sensor \n\nEnergy Sensor \n\nMouth Open \n\nFigure 2:  Consummatory Controller \n\nand  the  bite  pacemaker  (BP).  The  net  effect  of these  excitatory  connections  is  a \npositive feedback loop:  biting movements excite FA,  which causes BP to cause more \nfrequent  biting  movements,  which  further  excites  FA  until  its  activity  saturates. \nThis neural positive feedback  loop is inspired by  work on the neural basis of feeding \narousal  maintenance in  Aplysia (Weiss,  Chiel,  Koch,  and Kupfermann,  1986). \nAs  the insect consumes food,  its energy level begins to rise.  This leads to increased \nactivity  in  ES  which  both  directly  inhibits  FA,  and  also  decreases  the gain  of the \npositive  feedback  loop  via  an  inhibitory  modulatory synapse  onto  the  connection \nbetween  MO  and  FA.  At  some  point,  these  inhibitory  effects  will  overcome  the \npositive feedback  and activity in  FA  will  drop  low enough  to terminate the feeding \nbehavior.  This  neural  mechanism  is  based  upon  a  similar  one  hypothesized  to \nunderlie satiation in  Aplysia (Weiss,  Chiel,  and Kupfermann,  1986). \n\n4  RESULTS \nWith  the  neural  controllers  described  above,  we  have found  that feeding  behavior \nin the artificial  insect exhibits four  of the six  characteristics of motivated behavior \nwhich  were  described  by  Kupfermann  (1974): \n\n\fNeural Implementation or Motivated Behavior:  Feeding in an Artificial Insect \n\n49 \n\nGrouping  and  sequencing  of behavior  in  time.  When  the  artificial  insect \nis  \"hungry\", it generates  appetitive  and  consummatory  behaviors  with  the  proper \nsequence,  timing,  and intensity in order to obtain food. \n\nGoal-Directedness.  Regardless  of its  environmental  situation,  a  hungry  insect \nwill  generate movements which  serve  to obtain food.  Therefore,  the  behavior  of a \nhungry insect  can only  be understood  by reference  to an  internal goal.  Due to the \ninternal effects of the energy sensor  (ES)  and feeding  arousal  (FA)  neurons on the \ncontrollers,  the insect's external stimuli are insufficient  to account for  its behavior. \n\nChanges  in  responsiveness  due  to  a  change  in  internal  state.  While  a \nhungry  insect  will  attempt  to  orient  to  and  consume  any  nearby  food,  a  satiated \none will ignore it.  In addition, once a hungry insect has consumed sufficient food,  it \nwill  simply  walk  over  the food  patch which  initially attracted  it.  We  will examine \nthe arousal and satiation of feeding  in this artificial insect in more  detail below. \nPersistence. If a hungry insect is removed from food  before it has fed  to satiation, \nits feeding  arousal will  persist,  and  it will  continue to exhibit feeding  movements. \n\nOne technique  that has been applied  to the study of feeding  arousal in natural an(cid:173)\nimals is  the examination of the time interval between successive  bites as an animal \nfeeds  under various conditions.  In  Aplysia, for  example,  the interbite interval  pro(cid:173)\ngressively  decreases  as  an  animal  begins  to  feed  (showing  a  build-up  of arousal), \nand increases as  the animal satiates.  In addition, the rate of rise and fall  of arousal \ndepends  upon  the  initial  degree  of satiation  (Susswein,  Weiss,  and  Kupfermann, \n1978). \nIn order to examine the role of feeding arousal in the artificial insect, we  performed \na  similar  set  of experiments.  Food  was  directly  presented  to  insects  with  differ(cid:173)\ning degrees  of initial  satiation,  and  the  time  interval  between  successive  bites  was \nrecorded  for  the  entire  resulting  consummatory  response.  Above  an  energy  level \nof  approximately  80%  of capacity,  insects  could  not  be  induced  to  bite.  Below \nthis  level,  however,  insects  began  to consume  the  food.  As  these  insects  fed,  the \ninterbite  interval  decreased  as  their  feeding  arousal  built  up  until  some  minimum \ninterval  was  achieved  (Figure  3).  The  rate  of build-up  of arousal  was  slowest  for \nthose  insects  with  the  highest  initial  degree  of satiation.  In  fact,  an  insect  whose \nenergy level was already 75% of capacity never  achieved  full  arousal.  As the feeding \ninsects  neared  satiation,  their  interbite interval  increased  as  arousal  waned.  It is \ninteresting  to  note  that,  regardless  of the  initial  degree  of satiation,  all  insects  in \nwhich  biting  was  triggered  fed  until  their  energy  stores  were  approximately  99% \nfull.  The appropriate number of bites to achieve this were generated in  all  cases. \nWhat  is  the  neural  basis of these  arousal  and satiation  phenomena?  Clearly,  the \nanswer  lies  in  the  interactions between  the internal energy sensor  and  the  positive \nfeedback  loop  mediated  by  the  feeding  arousal  neuron,  but  the  precise  nature  of \nthe  interaction  is  not  at  all  clear  from  the  qualitative  descriptions  of the  neural \ncontrollers  given  earlier.  In  order  to  more  carefully  examine  this  interaction,  we \nproduced  a  phase plot of the activity in these  two neurons under the experimental \n\n\f50 \n\nBeer and Chiel \n\n400 \n\n500 -u \nCD en \nE \n....... -ca \n-.5 \nt= \nCD \n-\n:s ... CD \n-.5 \n\n300 \n\n100 \n\nCD \n\n200 \n\n.. \n.. \n\n25% Satiation \n\n- 50% Satiation \n\n60% Satiation \n75'Y. Satiation \n\nQ \n\n0 \n\n0 \n\n10 \n\n20 \n30 \nBite Number \n\n40 \n\n50 \n\nFigure 3:  Build-Up of Arousal  and  Satiation \n\nconditions described  above  (Figure 4). \n\nAn insect with a full  complement of energy begins at the lower right-hand corner of \nthe  diagram,  with  maximum activity  in  ES  and  no  activity  in  FA.  As  the  insect's \nenergy  begins  to fall,  it moves  to  the left  on  the  ES  axis  until the inhibition from \nES  is  insufficient  to hold  FA  below  threshold.  At this point,  activity in  FA  begins \nto increase.  Since the positive feedback loop is  not yet active because no biting has \noccurred,  a  linear  decrease  in energy  results  in  a  linear  increase  in  FA  activity.  If \nno food is  consumed,  the insect continues to move along this line  toward the upper \nleft of the diagram until its energy is exhausted. \n\nHowever, if biting is triggered by the presence of food  at the mouth, the relationship \nbetween  FA  and  ES  changes  drastically.  As  the  insect  begins  consuming  food, \nactivity in  FA  initially  increases  as  arousal  builds  up,  and  then later  decreases  as \nthe insect satiates.  Each  \"bump\"  corresponds  to the arousing effects on FA  of one \nbite via the positive feedback loop and to the small increase of energy from the food \nconsumed in that bite.  Trajectories are shown for  energy levels of 25%,  50%,  60%, \n65%,  and 75% of capacity.  The shape of these trajectories depend upon the activity \nlevel  of  FA  and  the  gain  of the  positive  feedback  loop  in  which  it  is  embedded, \nboth  of which  in  turn  depend  upon  the negative  feedback  from  the energy  sensor. \nWe  must  therefore  conclude  that,  even  in  this  simple  artificial  insect,  there  is  no \nsingle neural correlate to  \"hunger\".  Instead, this motivational state is  the result of \nthe  complex  dynamics  of interaction  between  the  feeding  arousal  neuron  and  the \ninternal energy sensor. \n\nReferences \n\nBeer,  R.  D.  (1989).  Intelligence  as  Adaptive  Behavior:  An  Experiment  in  Compu(cid:173)\ntational  Neuroethology.  Ph.D.  Dissertation,  Dept.  of Computer  Engineering  and \nScience,  Case  Western  lleserve  University.  Also  available  as  Technical  Report  TR \n\n\fNeural Implementation of Motivated Behavior:  Feeding in an Artificial Insect \n\nSI \n\n> \n\n~ > .-..... o \u00ab \n\u00ab u. \n\nES Activity \n\nFigure 4:  Phase  Plot of FA  vs.  ES  Activity \n\n89-118,  Center for  Automation and Intelligent Systems Research. \nBeer,  R.  D.,  Chiel,  H.  J.  and  Sterling,  L.  S.  (1989).  Heterogeneous  Neural  Net(cid:173)\nworks  for  Adaptive Behavior  in  Dynamic  Environments.  In  D.S.  Touretzky (Ed.), \nAdvances  in  Neural Information  Processing  Systems  1 (pp.  577-585).  San  Mateo, \nCA:  Morgan  Kaufmann  Publishers. \nChiel,  H.  J.  and  Beer,  R.  D.  (1989).  A  lesion  study  of a  heterogeneous  neural \nnetwork for  hexapod locomotion.  Proceedings  of the International Joint  Conference \non  Neural  Networks (IJCNN 89),  pp.  407-414. \nKupfermann,  I.  J.  (1974).  Feeding  behavior  in  Aplysia:  A  simple  system for  the \nstudy of motivation.  Behavioral Biology 10:1-26. \nSusswein, A.  J., Weiss,  K.  R. and Kupfermann, 1. (1978).  The effects of food  arousal \non the latency of biting in  Aplysia.  J.  Compo  Physiol.  123:31-41. \nWeiss,  K.  R.,  Chiel,  II.  J.,  Koch,  U.  and  Kupfermann,  1.  (1986).  Activity  of an \nidentified  histaminergic  neuron,  and its possible role  in  arousal of feeding  behavior \nin semi-intact  Aplysia.  J.  Neuroscience 6(8):2403-2415. \nWeiss,  K.  R.,  Chiel,  II.  J.  and  Kupfermann,  I. (1986).  Sensory function  and gating \nof histaminergic  neuron C2  in  Aplysia.  J.  Neuroscience 6(8):2416-2426. \n\n\f", "award": [], "sourceid": 254, "authors": [{"given_name": "Randall", "family_name": "Beer", "institution": null}, {"given_name": "Hillel", "family_name": "Chiel", "institution": null}]}