{"title": "Neural Network Routing for Random Multistage Interconnection Networks", "book": "Advances in Neural Information Processing Systems", "page_first": 722, "page_last": 729, "abstract": null, "full_text": "Neural Network Routing for  Random Multistage \n\nInterconnection Networks \n\nMark W.  Goudreau \nPrinceton University \n\nand \n\nNEe Research  Institute,  Inc. \n\n4 Independence  Way \nPrinceton,  NJ  08540 \n\nc. Lee  Giles \n\nNEC  Research  Institute, Inc. \n\n4 Independence  Way \nPrinceton,  NJ  08540 \n\nAbstract \n\nA routing scheme that uses  a neural network has been developed  that can \naid  in  establishing  point-to-point  communication  routes  through  multi(cid:173)\nstage interconnection  networks  (MINs).  The neural network  is  a  network \nof the  type  that  was  examined  by  Hopfield  (Hopfield,  1984  and  1985). \nIn  this  work,  the  problem  of establishing  routes  through  random  MINs \n(RMINs)  in a shared-memory, distributed computing system is  addressed. \nThe performance of the neural network routing scheme is  compared to two \nmore traditional approaches - exhaustive search  routing  and greedy  rout(cid:173)\ning.  The results suggest  that a neural network  router may be  competitive \nfor  certain RMIN s. \n\n1 \n\nINTRODUCTION \n\nA  neural  network  has  been  developed  that  can  aid  in  establishing  point-to(cid:173)\npoint  communication routes  through  multistage interconnection  networks  (MINs) \n(Goudreau and Giles,  1991).  Such interconnection networks  have been widely stud(cid:173)\nied  (Huang,  1984;  Siegel,  1990).  The  routing  problem  is  of great  interest  due  to \nits broad applicability.  Although the neural network  routing scheme can accommo(cid:173)\ndate  many types of communication systems,  this work  concentrates  on  its use  in  a \nshared-memory, distributed  computing system. \nNeural networks have sometimes been used to solve certain interconnection network \n\n722 \n\n\fNeural  Network Routing for Random Multistage Interconnection Networks \n\n723 \n\nInput \nPorts \n\nOutput \nPorts \n\nInterconnection \n\nNetwork \n\nControl Bits \n-\n\n:1 \n\n-\n\nI \n:  Logic1 \nLogic2: \nL.:  _  _  _  _  _  _  _  .... _-_-:--r-_~_ _ _ _'--_ -_-_ -_-J:.J \nI \nI \n\nNeural \nNetwork \n\nInterconnection \n\nNetwork \nController \n\nExterna  Control \n\nFigure  1:  The  communication system  with  a  neural  network  router.  The  input \nports (processors)  are on the left,  while the output ports (memory modules) are  on \nthe right. \n\nproblems,  such  as  finding  legal  routes  (Brown,  1989;  Hakim  and  Meadows,  1990) \nand increasing the throughput of an interconnection network (Brown and Liu, 1990; \nMarrakchi  and  Troudet,  1989).  The  neural  network  router  that  is  the  subject  of \nthis  work,  however,  differs  significantly  from  these  other  routers  and  is  specially \ndesigned  to handle parallel processing systems that have  MINs  with random inter(cid:173)\nstage  connections.  Such  random  MINs  are  called  RMINs.  RMINs  tend  to  have \ngreater fault-tolerance  than regular  MINs. \nThe  problem  is  to  allow  a  set  of  processors  to  access  a  set  of  memory  modules \nthrough the RMIN. A picture of the communication system with the neural network \nrouter  is  shown  in  Figure  1.  The  are  m  processors  and  n  memory  modules.  The \nsystem  is  assumed  to  be  synchronous.  At  the  beginning  of a  message  cycle,  some \nset  of  processors  may  desire  to  access  some  set  of  memory  modules. \nIt  is  the \njob  of the  router  to  establish  as  many  of these  desired  connections  as  possible  in \na  non-conflicting  manner.  Obtaining  the  optimal solution is  not  critical.  Stymied \nprocessors  may attempt communication again during the subsequent message cycle. \nIt is  the  combination of speed  and  the quality of the solution that is important. \nThe object of this work was to discover if the neural network router could be compet(cid:173)\nitive with other types of routers  in terms of quality  of solution,  speed,  and  resource \n\n\f724 \n\nGoudreau and Giles \n\nRMINI \n\n2 \n\n3 \n\n1 \n\nRMIN2 \n\n4 \n5 \n6 \n\nRMIN3 \n2 \n3 \n\n4 \n\n1 \n\n1 \n2 \n3 \n\n4 \n\n3 \n\n7 \n8 \n\n1 \n2 \n3 \n\n4 \n5 \n6 \n7 \n8 \n9 \n10 \n\n1 \n2 \n\n3 \n4 \n5 \n6 \n\n7 \n8 \n\nFigure 2:  Three random multistage interconnection  networks.  The blocks  that  are \nshown are crossbar switches, for which each input may be connected to each output. \n\nutilization.  To this  end,  the  neural  network  routing scheme  was  compared  to two \nother schemes for routing in RMINs - namely, exhaustive search routing and greedy \nrouting.  So  far,  the  results  of this  investigation suggest  that  the  neural  network \nrouter may indeed  be  a  practicable  alternative  for  routing in  RMINs  that are  not \ntoo large. \n\n2  EXHAUSTIVE SEARCH ROUTING \n\nThe exhaustive search routing method is optimal in terms of the ability of the router \nto  find  the  best solution.  There  are  many ways  to implement such  a  router.  One \napproach is  described  here. \n\nFor  a  given  interconnection  network,  every  route  from  each  input  to  each  output \nwas  stored  in  a  database.  (The  RMIN s  that  were  used  as  test  cases  in  this  paper \nalways had at least one route from each processor  to each memory module.)  When \na  new  message  cycle  began  and  a  new  message  set  was  presented  to  the  router, \nthe  router  would  search  through  the  database  for  a  combination of routes  for  the \nmessage  set  that  had  no  conflicts.  A  conflict  was  said  to  occur  if more  than  one \nroute  in the set  of routes  used  a single  bus in  the interconnection  network.  In the \ncase where every combination of routes for the message set had a conflict, the router \nwould find  a combination of routes that could establish the largest possible number \nof desired  connections. \nIf there  are  k  possible  routes  for  each  message,  this  algorithm needs  a  memory of \nsize 8( mnk) and, in the worst  case,  takes exponential time with respect  to the size \n\n\fNeural  Network Routing for Random Multistage Interconnection Networks \n\n725 \n\nof the  message  set.  Consequently,  it  is  an  impractical  approach  for  most  RMINs, \nbut it provides a convenient  upper bound for  the performance of other routers. \n\n3  GREEDY ROUTING \n\nWhen greedy routing is applied, message connections are established one at a time. \nOnce a route is established in a given message cycle, it may not be removed.  Greedy \nrouting does  not  always provide the optimal routing solution. \nThe greedy  routing  algorithm that  was  used  required  the  same route  database  as \nthe  exhaustive  search  router  did.  However,  it  selects  a  combination of routes  in \nthe following manner.  When a  new  message  set  is  present,  the  router  chooses  one \ndesired  message  and  looks  at the  first  route  on  that message's  list  of routes.  The \nrouter  then  establishes  that  route.  Next,  the  router  examines  a  second  message \n(assuming  a  second  desired  message  was  requested)  and  sees  if one  of the  routes \nin  the  second  message's  route  list  can  be  established  without  conflicting  with  the \nalready  established  first  message.  If such  a  route does  exist,  the router establishes \nthat route  and moves on to the next  desired  message. \nIn  the  worst  case,  the  speed  of the  greedy  router is  quadratic  with respect  to the \nsize  of the message set. \n\n4  NEURAL NETWORK ROUTING \n\nThe focal  point  of the  neural  network  router  is  a  neural  network  of the  type  that \nwas  examined by  Hopfield  (Hopfield,  1984 and  1985).  The problem of establishing \na set of non-conflicting routes can  be reduced  to a constraint satisfaction problem. \nThe structure of the neural network  router is  completely determined by the RMIN. \nWhen a new set of routes is desired, only certain bias currents in the network change. \nThe  neural  network  routing scheme  also  has  certain  fault-tolerant  properties  that \nwill not be described  here. \nThe neural network  calculates the routes by  converging to a legal  routing  array.  A \nlegal  routing  array  is  3-dimensional.  Therefore,  each  element  of the  routing  array \nwill  have  three  indices.  If element  ai,i,k  is  equal  to  1  then  message  i  is  routed \nthrough  output  port  k  of stage  j.  We  say  ai,;,k  and  a',m,n  are  in  the  same  row if \ni = I and k = n.  They are in  the same  column if i = I and j  = m.  Finally, they are \nin the same  rod if j  = m  and k  = n. \nA legal routing array will satisfy  the following three  constraints: \n\n1.  one  and only one element in each column is equal to  1. \n2.  the elements  in successive  columns that  are  equal  to  1 represent  output  ports \n\nthat can be connected  in  the interconnection  network. \n\n3.  no  more than one  element  in each  rod is  equal to  1. \n\nThe  first  restriction  ensures  that  each  message  will  be  routed  through  one  and \nonly  one  output  port  at  each  stage  of the  interconnection  network.  The  second \nrestriction  ensures  that  each  message  will  be  routed  through  a  legal  path  in  the \n\n\f726 \n\nGoudreau and Giles \n\ninterconnection network.  The third restriction ensures  that any resource  contention \nin  the  interconnection  network  is  resolved.  In  other  words,  only  one  message  can \nuse  a certain output port  at a  certain stage in the interconnection  network.  When \nall  three  of these  constraints  are  met,  the  routing  array  will  provide  a legal  route \nfor  each  message in the message set. \nLike the routing array, the neural network router will naturally have a 3-dimensional \nstructure.  Each  ai,j,k  of a  routing  array  is  represented  by  the  output  voltage of a \nneuron,  V'i,j,k'  At  the  beginning  of a  message  cycle,  the  neurons  have  a  random \noutput  voltage.  If the  neural  network  settles  in  one  of  the  global  minima,  the \nproblem will  have  been solved. \nA continuous time mode network was chosen.  It was simulated digitally.  The neural \nnetwork has N  neurons.  The input to neuron i is  Ui, its input bias current is  Ii, and \nits output is  Vi.  The input Ui  is  converted  to the output Vi  by  a sigmoid function, \ng(z).  Neuron  i  influences  neuron  j  by  a  connection  represented  by 7ji.  Similarly, \nneuron j  affects neuron i through connection Iij.  In order for the Liapunov function \n(Equation 5) to be constructed, Iij must equal7ji.  We further assume that Iii = O. \nFor  the  synchronous  updating model, there  is  also  a time constant,  denoted by  T. \nThe equations which  describe  the output of a  neuron  i  are: \n\nU\u00b7  LN \n\nduo \n-' = --' + \ndt \n\nT \n\n.  1 \nJ= \n\nT. .. v,.  + L\u00b7 \n, \n~  J \n\nT=RC \nV;  = g(Uj) \n1 \n\ng(z) =  1 + e-X \n\n(1) \n\n(2) \n(3) \n\n(4) \n\nThe equations above force  the neural net into stable states that are the local minima \nof this approximate energy equation \n\niNN  \n\nE  = - 2 L 2: Iij Vi V;  - L V'i Ii \n\nN \n\ni=1j=1 \n\ni=l \n\n(5) \n\nFor  the  neural  network,  the  weights  (Iii's)  are  set,  as  are  the  bias  currents  (Ii'S). \nIt is  the output voltages  (V'i's)  that vary to to minimize E. \nLet  M  be  the number of messages  in a  message set,  let  S  be  the  number of stages \nin  the  RMIN,  and  let  P  be  the  number  of ports  per  stage  (P  may be  a  function \nof the  stage  number).  Below  are  the  energy  functions  that  implement  the  three \nconstraints  discussed  above: \n\nA  M  8-1  P \n\nE1  = 2' 2: L 2: Vm\",p(-Vm\",p + 2: Vm,3,i) \n\nE2 = 2' 2: 2: 2: Vm,I,p( - Vm,3,p + L V'i,3,p) \n\nm=1  1=1 p=l \nB  8-1  P  M \n\n,=1 p=1 m=1 \n\nP \n\ni=1 \n\nM \n\ni=1 \n\n(6) \n\n(7) \n\n\fNeural  Network Routing for Random Multistage Interconnection Networks \n\n727 \n\nC  M  S-l  P \n\nEa  = \"2 2: 2: 2:( -2Vm\",p + Vm\",p(-Vm\",p + 2: Vm\",i)) \n\nP \n\ni=l \n\nm=l ,=1 p=l \n\nM  [S-l  P  P \n\nD f.  ~]; tt d(s, p, i)Vm,,-l,p Vm\",i \n+ &,( d( 1, (JIm, j)Vm,IJ + d( S, j, Pm )Vm,S -IJ )] \n\n(8) \n\n(9) \n\nA,  B,  C,  and  D  are  arbitrary  positive  constants. l  El  and  Ea  handle  the  first \nconstraint in the routing array.  E4  deals with the second constraint.  E2  ensures the \nthird.  From the equation for E4,  the function d(sl,pl,p2) represents  the \"distance\" \nbetween  output port pI from stage sl - 1 and output port p2  from stage s1.  If pI \ncan  connect  to  p2  through stage  sl,  then  this  distance  may  be  set  to  zero.  If pI \nand  p2  are  not  connected  through  stage  sl,  then  the  distance  may be  set  to  one. \nAlso,  am  is  the  source  address  of message  m,  while  f3m  is  the  destination  address \nof message  m. \n\nThe entire energy function  is: \n\n(10) \n\nSolving for  the  connection  and  bias  current  values  as  shown in  Equation  5 results \nin the following equations: \n\n(11) \n\n-B031 ,,20pl,p2(1  - Oml,m2) \n-D8m1,m2[031+1,,2d(s2,pl,p2) + 8,1,,2+1 d(sl,p2,pl)] \n(12) \n\n1m \",p = C - D[8\"ld(l, am,p) + o\"s-ld(S,p,f3m)] \n\n8i,j  is  a  Kronecker  delta (8j,j  = 1 when  i  = j, and 0 otherwise). \nEssentially,  this  approach  is  promising because  the  neural  network  is  acting  as  a \nparallel computer.  The hope is that the neural network will generate solutions much \nfaster  than conventional approaches for  routing in  RMINs. \nThe neural  network  that is  used  here  has  the  standard  problem - namely, a global \nminimum is  not  always  reached.  But  this  is  not  a  serious  difficulty.  Typically, \nwhen  the  globally minimal energy  is  not  reached  by  the  neural  network,  some  of \nthe  desired  routes  will  have  been  calculated  while  others  will  not  have.  Even  a \nlocally  minimal solution  may  partially solve  the  routing  problem.  Consequently, \nthis  would  seem  to be  a  particularly encouraging  type  of application for  this type \nof neural  network.  For  this  application,  the  traditional  problem  of not  reaching \nthe global minimum may not  hurt  the system's  performance  very  much,  while  the \nexpected  speed  of the  neural  network  in  calculating  the  solution  will  be  a  great \nasset. \n\nIFor  the simulations,  T  = 1.0,  A = 0  = D = 3.0,  and  B  =  6.0.  These  values  for  A, B, \n\n0, and  D  were  chosen  empirically. \n\n\f728 \n\nGoudreau and Giles \n\nTable 1:  Routing results for  the RMINs shown  in  Figure 2.  The * entries  were  not \ncalculated due  to their computational complexity. \n\nM \n1 \n2 \n3 \n4 \n5 \n6 \n7 \n8 \n\nRMIN1 \n\nRMIN2 \n\nRMIN3 \n\nEgr  Enn \nEel \n1.00  1.00  1.00 \n1.86  1.83  1.87 \n2.54  2.48  2.51 \n3.08  2.98  2.98 \n3.53  3.38  3.24 \n3.89  3.67  3.45 \n4.16  3.91  3.66 \n4.33  4.10  3.78 \n\n2.91 \n\nEgr  Enn \nEel \n1.00  1.00  1.00 \n1.97  1.97  1.98 \n2.91 \n2.93 \n3.80  3.79  3.80 \n4.65  4.62  4.61 \n5.44  5.39  5.36 \n6.17  6.13  6.13 \n6.86  6.82  6.80 \n\n2.71 \n\nEgr  Enn \nEel \n1.00  1.00  1.00 \n1.99  1.88  1.94 \n2.99 \n2.87 \n3.94  3.49  3.72 \n*  4.22  4.54 \n*  4.90  5.23 \n*  5.52  5.80 \n*  6.10  6.06 \n\nThe  neural  network  router  uses  a  large  number  of neurons.  If there  are  m  input \nports,  and  m  output  ports  for  each  stage  of the  RMIN,  an  upper  bound  on  the \nnumber of neurons needed  is  m2 S.  Often, however,  the number of neurons  actually \nrequired is  much smaller than this upper  bound. \nIt has been  shown empirically that neural networks of the  type used  here  can con(cid:173)\nverge to a solution in essentially constant time.  For example, this claim is made for \nthe neural network described  in (Takefuji and Lee,  1991), which is a slight variation \nof the  model used  here. \n\n5  SIMULATION  RESULTS \n\nFigure 2 shows three  RMINs that were  examined.  The routing results for  the three \nrouting  schemes  are  shown  in  Table  1.  Eel  represents  the  expected  number  of \nmessages  to  be  routed  using  exhaustive  search  routing.  Egr  is  for  greedy  routing \nwhile  Enn  is  for  neural  network  routing.  These  values  are  functions  of  the  size \nof  the  message  set,  M.  Only  message  sets  that  did  not  have  obvious  conflicts \nwere  examined.  For  example,  no  message  set  could  have  two  processors  trying  to \ncommunicate to the same memory module.  The table shows  that, for  at least  these \nthree RMINs, the three routing schemes produce solutions that are of similar virtue. \nIn  some  cases,  the  neural  network  router  appears  to  outperform  the  supposedly \noptimal exhaustive  search  router.  That  is  because  the  Eel  and  Egr  values  were \ncalculated  by  testing  every  message  set  of size  M,  while  Enn  was  calculated  by \ntesting  1,000  randomly  generated  message  sets  of size  M.  For  the  neural  network \nrouter to appear to perform best,  it must have gotten message sets that were  easier \nto route  than average. \nIn general, the performance of the neural network  router degenerates  as  the size  of \nthe RMIN increases.  It is felt  that the neural network router in its present form will \nnot  scale  well  for  large  RMINs.  This  is  because  other  work  has  shown  that  large \nneural  networks of the type used  here  have  difficulty converging  to a  valid solution \n(Hopfield,  1985). \n\n\fNeural  Network Routing for  Random  Multistage Interconnection Networks \n\n729 \n\n6  CONCLUSIONS \n\nThe results show that there is not much difference, in terms of quality of solution, for \nthe  three  routing  methodologies working on  these  relatively small sample RMINs. \nThe  exhaustive  search  approach  is  clearly  not  a  practical  approach  since  it is  too \ntime  consuming.  But  when  considering  the  asymptotic  analyses  for  these  three \nmethodologies one  should keep  in mind the performance degradation of the greedy \nrouter and the  neural network  router  as  the size  of the  RMIN  increases. \nGreedy  routing  and  neural  network  routing  would  appear  to  be  valid  approaches \nfor  RMINs  of moderate  size.  But  since  asymptotic  analysis  has  a  very  limited \nsignificance here,  the  best way  to compare the speeds of these  two  routing schemes \nwould be  to build  actual implementations. \nSince  the neural  network  router  essentially  calculates  the routes  in  parallel,  it  can \nreasonably  be  hoped  that  a  fast,  analog  implementation  for  the  neural  network \nrouter  may find  solutions  faster  than  the  exhaustive  search  router  and  even  the \ngreedy  router.  Thus,  the  neural  network  router  may  be  a  viable  alternative  for \nRMIN s that are  not  too large. \n\nReferences \n\nBrown, T. X.,  (1989),  \"Neural networks for switching,\"  IEEE Commun.  Mag.,  Vol. \n27,  pp.  72-81,  Nov.  1989. \nBrown,  T. X.  and Liu,  K.  H.,  (1990),  \"Neural network  design  of a banyan network \ncontroller,\"  IEEE  J.  on  Selected  Areas of Comm., pp.  1428-1438, Oct.  1990. \nGoudreau,  M.  W.  and  Giles,  C.  L.,  (1991),  \"Neural  network  routing  for  multiple \nstage interconnection  networks,\"  Proc.  IJCNN 91,  Vol.  II,  p.  A-885, July 1991. \nHakim, N.  Z.  and Meadows,  H.  E., (1990),  \"A neural network approach to the setup \nof the Benes switch,\"  in  Infocom  90,  pp.  397-402. \nHopfield, J. J., (1984),  \"Neurons with graded response have collective computational \nproperties  like  those  of two-state  neurons,\"  Proc.  Natl.  Acad.  Sci.  USA,  Vol.  81, \npp.  3088-3092, May  1984. \nHopfield, J. J ., (1985), \"Neural computation on decisions in optimization problems,\" \nBioi.  Cybern.,  Vol.  52,  pp.  141-152, 1985. \nHuang, K.  and Briggs, F. A., (1984),  Computer Architecture and Parallel Processing, \nMcGraw-Hill, New  York,  1984. \nMarrakchi, A.  M.  and Troudet, T., (1989),  \"A  neural net  arbitrator for large cross(cid:173)\nbar packet-switches,\"  IEEE Trans.  on  Cire.  and Sys.,  Vol.  36, pp.  1039-1041, July \n1989. \nSiegel,  H.  J., (1990),  Interconnection  Networks for Large  Scale  Parallel Processing, \nMcGraw-Hill, New  York,  1990. \nTakefuji, Y. and Lee,  K.  C., (1991),  \"An artificial hysteresis binary neuron:  a model \nsuppressing  the  oscillatory  behaviors  of neural  dynamics\",  Biological  Cybernetics, \nVol.  64,  pp.  353-356,  1991. \n\n\f", "award": [], "sourceid": 516, "authors": [{"given_name": "Mark", "family_name": "Goudreau", "institution": null}, {"given_name": "C.", "family_name": "Giles", "institution": null}]}