{"title": "Comparison Training for a Rescheduling Problem in Neural Networks", "book": "Advances in Neural Information Processing Systems", "page_first": 801, "page_last": 808, "abstract": null, "full_text": "Comparisoll Training  for  a  Resclleduling \n\nProblem ill  Neural  Networks \n\nDidier  Keymeulen \n\nArtificial Intelligence  Laboratory \n\nVrije  Universiteit  Brussel \nPleinlaan  2,  1050  Brussels \n\nBelgium \n\nMartine de  Gerlache \n\nProg Laboratory \n\nVrije  Universiteit  Brussel \nPleinlaan  2,  1050  Brussels \n\nBelgium \n\nAbstract \n\nAirline  companies  usually  schedule  their flights  and  crews  well  in \nadvance  to optimize  their  crew  pools  activities.  Many  events such \nas  flight  delays or  the  absence  of a  member require  the  crew  pool \nrescheduling team to change the initial schedule  (rescheduling).  In \nthis paper, we  show  that the neural network comparison paradigm \napplied  to  the  backgammon  game  by  Tesauro  (Tesauro  and  Se(cid:173)\njnowski,  1989)  can  also  be applied  to the rescheduling  problem of \nan  aircrew  pool. \nIndeed  both  problems  correspond  to  choosing \nthe best solut.ion  from a set of possible ones without ranking them \n(called here best choice problem).  The paper explains from a math(cid:173)\nematical point of view  the architecture and the learning strategy of \nthe backpropagation neural network used for  the best  choice  prob(cid:173)\nlem.  We  also  show  how  the learning  phase  of the  network  can  be \naccelerated.  Finally  we  apply  the  neural  network  model  to some \nreal rescheduling  problems for  the  Belgian  Airline  (Sabena). \n\n1 \n\nIntroduction \n\nDue  to merges,  reorganizations and  the need  for  cost  reduction,  airline  companies \nneed  to  improve  the  efficiency  of their  manpower  by  optimizing  the  activities  of \ntheir crew  pools  as  much  as  possible.  A st.andard scheduling of flights  and  crews is \nusually  made well  in  advance but many events, such  as flight  delays or the absence \nof a  crew  member make  many schedule  cha.nges  (rescheduling)  necessary. \n\n801 \n\n\f802 \n\nKeymeulen and de Gerlache \n\nEach  day,  the  CPR 1  team  of  an  airline  company  has  to  deal  with  these  pertur(cid:173)\nbations.  The  problem  is  to  provide  the  best  answer  to  these  regularly  occurring \nperturbations and to limit their impact on  the general schedule.  Its solution is hard \nto find  and usually the CPR team calls on full  reserve crews.  An efficient  reschedul(cid:173)\ning  tool  taking into account  the  experiences  of the  CPR team  could  substantially \nreduce  the  costs  involved  in  rescheduling  notably  by  limit.ing  the  use  of a  reserve \ncrew. \n\nThe paper is organized as follow.  In the second section we  describe  the rescheduling \ntask.  In the third section we  argue for  the use of a neural network for  the reschedul(cid:173)\ning  task  and  we  apply  an  adequate  architecture  for  such  a  network.  Finally  in \nthe  last  section,  we  present  results  of experiments  with  schedules  based  on  actual \nschedules  used  by  Sabena. \n\n2  Rescheduling for  an  Airline  Crew Pool \n\nWhen  a  pilot  is  unavailable  for  a  flight  it  becomes  necessary  to  replace  him,  e.g. \nto reschedule  the  crew.  The  rescheduling  starts from  a  list  of potential substitute \npilots (PSP) given by a scheduling program based generally on operation research or \nexpert syst.em technology  (Steels,  1990).  The PSP list obtained respects legislation \nand  security  rules  fixing  for  example  t.he  number  of flying  hours  per  month,  the \nmaximum  number  of consecutive  working  hour  and  the  number  of training  hours \nper  year  and  t.heir  schedule.  From  the  PSP  list,  the  CPR  team  selects  the  best \ncandidat.es  taking  into  account  t.he  schedule  stability  and  equity.  The  schedule \nstability requires that possible perturbations of the schedule  can be dealt with  with \nonly a minimal rescheduling effort.  This criterion ensures work stability t.o  the crew \nmembers  and  has  an  important  influence  on  their  social  behavior.  The  schedule \nequity  ensures  the  equal  dist.ribution  of  the  work  and  payment  among  the  crew \nmembers  during the schedule  period. \n\nOne may think to solve this rescheduling problem in  t.he  same way as the scheduling \nproblem itself using  software  t.ools  based  on  operational research  or expert system \napproach.  But  t.his  is  inefficient.  for  t.wo  reasons,  first.,  the scheduling issued  from  a \nscheduling system and its adapt.at.ion  t.o  obt.ain  an  acceptable schedule  takes  days. \nSecond  this  system  does  not  t.ake  into  account  the  previous  schedule.  It  follows \nthat  the  updat.ed  one  may  differ  significantly  from  the  previous  one  after  each \nperturbation.  This  is  unaccept.able  from  a  pilot's  point  of view.  Hence  a  specific \nprocedure for  rescheduling  is  necessary. \n\n3  Neural Network  Approach \n\nThe  problem of reassigning  a  new  crew  member  to replace  a  missing  member  can \nbe  seen  as  the  problem  of finding  the  best  pilot  in  a  pool  of  potential  substitute \npilots  (PSP), called  the  best  choice  problem. \n\nTo solve  the  best  choice  problem,  we  choose  the  neural  network  approach  for  two \nreasons.  First the rules llsed by the expert. are not well defined:  to find  the best PSP, \n\nlCrew  Pool Rescheduler \n\n\fComparison Training for a Rescheduling Problem in  Neural Networks \n\n803 \n\nthe expert associates implicit.ly  a score value  to each  profile.  The learning approach \nis  precisely well suited to integrate,  in  a short period of time,  t.he  expert knowledge \ngiven  in  an  implicit  form.  Second,  t.he  neural  network  approach  was  applied  with \nsuccess to board-games e.g.  the Backgammon game described  by  Tesauro (Tesauro \nand Sejnowski,  1989)  and  the  Nine  l\\llen's Morris game described  by  Braun (Braun \nand  al.,  1991).  These  two  games  are  also  exa.mples  of best  choice  problem  where \nthe player  chooses  the  best move  from  a  set of possible  ones. \n\n3.1  Profile of a  Potential Substitute Pilot \n\nTo  be  able  to  use  the  neural  network  approach  we  have  to  identify  the  main  fea(cid:173)\ntures of the potential substitute pilot  and  to  codify  them in  terms of rating values \n(de  Gerlache  and  Keymeulen,  1993).  We  based our  coding  scheme  on  the way  the \nexpert solves  a  rescheduling  problem.  He  ident.ifies  the relevant  parameters associ(cid:173)\nated  with  the PSP  and  the perturbed schedule.  These parameters give  three types \nof information.  A  first  type  describes  the previous,  present  and  future  occupation \nof the  PSP.  The  second  t.ype  represents  information  not  in  the  schedule  such  as \nthe  human  relationship  fadars.  The  assocjat.ed  values  of t.hese  two  t.ypes  of pa(cid:173)\nrameters  differ  for  f'ach  PSP.  The  last  t.ype  of paramet.ers  describes  the  context \nof the  rescheduling,  namely  t.he  characteristics  of t.he  schedule .  This  last  type  of \nparameters  are  the  same  for  all  the  PSP.  All  t.hese  paramet.ers form  the  profile  of \na  PSP  associated  to  a  perturbed  schedule.  At  each  rescheduling  problem  corre(cid:173)\nsponds  one  perturbed  schedule  j  and  a  group  of 11  PSpi  to  which  we  associate  a \n\nProjile~ = (PSpi, PertU1\u00b7berLSchedulej) .  Implicitly,  the  expert  associates  a  rat-\n\ning value  between  0 and  1 to each  parameter of the  P1'ojile;  based on  respectively \nits  little  or  important  impact  on  the  result.ing  schedule  if the  P S pi  was  chosen. \nThe rating value  reflects  the  relative  importance of the  parameters on  the stability \nand  the equity of the resulting schf'dnle  obt.ained  after  the  pilots  substitution. \n\n3.2  Dual Neural  Network \n\nIt would  have  been possible  to get  more information from  the expert  than only  the \nbest  profile.  One of the  possibilities  is  to  ask  him  to score  every  profile  associated \nwith  a  perturbed  planning.  From  this  associat.ion  we  could  immediately  construct \na  scoring function  which couples each profile with  a specific value,  namely its score. \nAnother  possibility  is  to ask  the  expert  to  rank  all  profiles  associated  with  a  per(cid:173)\nturbed  schedule.  The  corresponding  ranking  function  couples  each  profile  with  a \nvalue such  that the values associat.ed  with  the profiles of the same perturbed sched(cid:173)\nule order the profiles  according t.o  t.heir  rank.  The decision  making process used  by \nthe  rescheduler  team  for  the  aircrew  rescheduling  problem  does  not  consist  in  the \nevaluation of a  scoring or  ranking function .  Indeed  only  the knowledge  of the  best \nprofile  is  useful  for  the rescheduling  process. \n\nFrom a neura.l net.work  architectural point of view,  because the ranking problem is a \ngeneralization of the best choice problem, a same neural net.work architecture can be \nused.  But the  difference  between  the  best  choice  problem and  t.he  scoring problem \nis  such  that two  different  neural  network  architectures  are  associated  to them.  As \nwe  show in  this section,  although  a backpropagatian network  is  sufficient  to learn a \nscoring  function,  its  architecture,  its  learning  and  its  retrieval  procedures must  be \n\n\f804 \n\nKeymeulen and de Gerlache \n\nadapted  to learn  the best profile.  Through  a  mathematical formulation  of the best \nchoice problem, we  show  that the comparison paradigm of Tesauro (Tesauro,  1989) \nis  suited  to  the  best  choice  problem  and  we  suggest  how  to  improve  the  learning \nconvergence. \n\n3.2.1  Comparing  Function \n\nFor the best choice problem the expert gives the best profile Projilefest associated \nwith  the  perturbed  schedule  j  and  that  for  m  pert.urbed  schedules.  The  problem \nconsists  then  to  learn  the  mapping  of  the  m  * n  profiles  associated  with  the  m \nperturbed schedules into the m  best profiles,  one for  each pert.urbed schedule.  One \nway  to  represent  this  association  is  through  a  comparing  function.  This function \nhas as input a profile,  represented by  a vector xj, and returns a single value.  When \na set of profiles associated with a  perturbed schedule are evaluated by  the function, \nit returns the lowest  value for  the  best  profile.  This comparing function  integrates \nthe  information  given  by  the  expert  and  is  sufficient  to  reschedule  any  perturbed \nschedule solved  in the past  by  the expert.  Formally it is  defined  by: \n\nComp(J.1>e)  = C(Projile)) \n\n(1) \n\nC \n\nomparej \n\nBest  C \n\n<  ompcl1>Cj \n\n,.  {V)' \n\nVi=fBest  with \n\nwith)' =  1, ... ,111. \ni=l, ... ,n \n\nThe value of Comp(J.1>e)  are  not known  a priori and have only  a meaning when they \nare  compared  to  the  value  Comp(J.1>ef est  of the  comparing  function  for  the  best \nprofile. \n\n3.2.2  Geometrical Interpretation \n\nTo illustrate the difference between the neural network learning of a scoring function \nand  a  comparing  function,  we  propose  a  geometrical  interpretation  in  the  case  of \na  linear  network  having  as  input  vect.ors  (profiles)  XJ, ... ,XJ, ... ,Xp  associated \nwith  a  perturbed schedule  j. \n\nThe learning of a scoring function  which  associat.es  a score  Score;  with each  input \nvector xj consists in  finding a hyperplane in the input vector space which is tangent \nto the circles of cent.er xf  and radius SC01>e{  (Fig.  1).  On the contrary the learning \nof a  comparing function  consists t.o  obt.ain  t.he  equation of an hyperplane such  that \nthe end-point  of the  vector  Xfest  is  nearer  the  hyperplane  than  the end-points  of \nthe other input vectors XJ  associated  with  the same perturbed schedule j  (Fig.  1). \n\n3.2.3  Learning \n\nWe  use  a  neural  network  approach  to build  the  comparing  function  and  the  mean \nsquared  error  as  a  measure  of  the  quality  of t.he  approximation.  The  comparing \nfunction  is  approximated  by  a  non-linear  function:  C(P1>ojile;)  =  N\u00a3(W,Xj) \nwhere W is  the weight.  vector of the neural network  (e.g backpropagat.ion network). \nThe problem of finding  C which  has  the property of (1)  is  equivalent  to finding  the \nfunction  C that minimizes the following  error function  (Braun  and  al.,  1991)  where \n<I>  is  the  sigmoid function  : \n\n\fComparison Training for a Rescheduling Problem in Neural Networks \n\n805 \n\n... --_ .. _(cid:173)\n\n,--\n\nx,, \n\n\"\"\"\", , , \n, \n\\ , , , , \nI , , , , , , , \n\nI \n\n,.,1' \n\n-(cid:173)\n\n.,' \n\nW( Wl ,w2'''')'  ..  ,WL) \nwlh \n\n.w. - 1 \n\nx..,,, \n\nI \n\" \n', ....... --,' \n\nFigure  1:  Geometrical  Interpretation  of  the  learning  of  a  Scoring  Function \n(Rigth)  and  a  Comparing Function  (Left) \n\nn \n\nI: \ni = 1 \ni  -::f  Best \n\n(2) \n\nTo  obtain  t.he  weight  vector  which  mll1UTIlzes  the  error  funct.ion  (2),  we  use  the \nproperty that  t.he  -gr~ld\u00a3~(W) point.s  in  the  direct.ion  in  which  the error function \nwill  decrease  at  the  fastest  possible  rate.  To  update  t.he  weight  we  have  thus  to \ncalculate  the  partial  derivative  of (2)  with  each  components  of the  weight  vector \nltV:  it is  made of a  product of three  factors.  The evaluation of the first  two  factors \n(the  sigmoid  and  the  derivative  of the  sigmoid)  is  immediate.  The  third  factor  is \nthe  partial  derivative  of the  non-linear  function  N \u00a3,  which  is  generally  calculated \nby using  the  generalized  delta  rule  learning law  (Rumelhart. and  McClelland,  1986), \n\nUnlike  the  linear  associator  network,  for  the  backpropagation  network,  the  error \nfunction  (2)  is  not equivalent  to the error function  where  the difference Xl e3t  - X; \nis  associated with  the input  vector of the  backpropagation  network because: \n\n(3) \n\nBy  consequence  to  calculate  t.he  three  factors  of  the  partial  derivative  of  (2),  we \nhave  to introduce separately at the bottom of the  network  t.he  input  vector  of the \nbest  profile X !e3t  and the  input  vector  of a  less  good  profile XJ.  Then  we  have  to \nmemorize theIl'  partial contribution  at each node of the network and multiply their \ncontributions before updating the weight .  Using this way  to evaluate the derivative \nof (2)  and to update t.he  weight,  the simplicity of the generalized delta rule learning \nlaw  has  disappeared . \n\n\f806 \n\nKeymeulen and de Gerlache \n\n3.2.4  Architecture \n\nTesauro  (Tesauro and  Sejnowski,  1989)  proposes  an  architecture,  t.hat  we  call  dual \nneural network, and a learning procedure such that the simplicity of the generalized \ndelta  rule  learning  law  can  still  be  used  (Fig.  2).  The  same  kind  of architecture, \ncalled  siamese  network,  was recently used  by  Bromley for  the signature verification \n(Bromley  and  al.,  1994).  The  dual  neural  network  architecture  and  the  learning \nstrategy are justified mathematically at one hand by the decomposition of the partial \nderivative of the error function  (2)  in  a sum of two terms and at the other hand by \nthe  asymmetry property of the sigmoid  and its derivative. \n\nThe  architecture  of the  dual  neural  network  consists  to  duplicate  the  multi-layer \nnetwork  approximating  the  comparing  function  (1)  and  to  connect  the  output  of \nboth to a unique output node through a positive unit weight for the left network and \nnegative  unit weight.  for  the right  network.  During the learning a  couple of profiles \nis presented to the dual neural network:  a best profile X f e3t  and a  less good  profile \nX!.  The desired  value  at  the output  node of the dual  neural  network is 0 when  the \nleft  network has for  input  the best profile  and the right network has for  input a less \ngood profile  and 1 when  these profiles are permuted.  During the recall we  work only \nwith  one  of the  two  multi-layer  networks,  suppose  the  left  one  (the  choice  is  of no \nimportance  because  they  are exactly  the same).  The  profiles  JY~  associated  with  a \nperturbed schedule j  are presented at the input of the left.  multi-layer network.  The \nbest  profile  is  the one  having  the  lowest  value  at  the output of the  left  multi-layer \nnetwork. \n\nThrough  this  mathematical  formulation  we  can  use  the  suggestion  of  Braun  to \nimprove  the  learning  convergence  (Braun  and  al.,  1991).  They  propose  to  replace \nthe  positive  and  negative  unit  weight  het.ween  the  output  node  of the  multi-layer \nnetworks  and  the output.  node  of the  dual  neural  network  by  respect.ively  a  weight \nvalue equal to V for  the left net.work  and - V for  the right.  network.  They modify the \nvalue of V by applying the generalized delt.a rule which  has  no significant impact on \nthe  learning  convergence.  By  manually increasing the factor  V  during  the  learning \nprocedure, we improve considerably the learning convergence  due to its asymmetric \nimpact on  the  derivative  of \u00a3<I>(W)  with  W:  the modification  of the  weight  vector \nis  greater for  couples  not yet  learned  than  for  couples already  learned. \n\n4  Results \n\nThe experiments show the abilit.y  of our model  to help  the CPR team of the Sabena \nBelgian  Airline  company  to  choose  the  best  profile  in  a  group  of  PSPs  based  on \nthe  learned  expertise  of  the  team.  To  codify  the  profile  we  identify  15  relevant \nparameters.  They  constitute  the  input of our  neural  network.  The  training  data \nset  was  obtained  by  analyzing  the  CPR team  at  work  during  15  days  from  which \nwe  retain our  training and  test perturbed schedules. \n\nWe  consider  that  the  network  has  learned  when  the  comparing  value  of the  best \nprofile is less than the comparing value of t.he  other profiles and that for  all  training \nperturbed schedules.  At that time \u00a3cJ>(W)  is less t.han  .5 for  every  couple of profiles. \nThe left  graph  of Figure  3 shows  t.he  evolution  of t.he  mean  error  over  t.he  couples \n\n\fComparison Training for a Rescheduling Problem in Neural Networks \n\n807 \n\nDual Neural Network \n\n\u00b71 \n\nC_ .. n \n\nBelt. \n\n.1Ir.(~.l \n\n) \nBeltoJ \n\nZMulI.La,'\"\" \n\n-'-od ..... \n\nNouol \nNoI\",ort.o \n\nC_,.n \n'J \n\n.11 r.(~. t .) \n\n,~ \n\n10.6lo.~ 10,910\", 10.110.710.11  XB .... I I63IO.2!O-.1  b.61  0.21  0.91  0.21  X2\u20221 \n10.310.210.11  0.61  0.21  0.91 0.21  X 2\u20221  10,616316.91  631  0.11  6.71  0.21  X \n\nB .... I \n\n-+ \n(X \n\n-+ \n.X \ntoJ \n\nBut~ \n\n) \n\n\u2022 \n\nwdb  XIJ  = X But.l \n\nit.t  ) 10.610.710.1/  0.11  0.711.01  0.21  XB ..... lo. 1Io.9Io.7!  0.910.110.11  0.&1  X  I .. \n,~  B.... \n10.610.710.1( 0.11  0.711.0 I 0.11  XB ..... \n\n10.11  0,91  0.71  0.91  0.11  0.11  0,61  X I.. \nWllb  'S ..  = X Be ... \n\nFigure 2:  The training of a  dual  neural  network. \n\nduring  the  training .  The  right  graph  shows  the  improvement  of the  convergence \nwhen  the weight  V  is  increased regularly  during  the training process. \n\nOJ \n\nOA \n\n0.1 \n\nlnclasilll V \n\nor the Dull NaIl'll Ndworll \n\no.s \n\n0 .\u2022 \n\nOJ \n\nM~--~~~----------------------\n\n02  ~--~~------------------------\n\n0.1 \n\n01 \n\nliIl) \n\nliIO \n\n111(1) \n\nIlO) \n\nNwnbcr \nof'lhilitg \n\nNunt.crci \n~~~~~~~~~~~~~~~~ nW~ \n\nFigure  3:  Convergence of the dual neural network architecture. \n\nThe  network  does  not  converge  when  we  introduce  contradictory  decisions  in  our \ntraining set.  It is  possible to resolve  them by  adding new context parameters in  the \ncoding scheme of the profile. \n\nAfter  learning,  our  network  shows  generalization  capacity  by  retrieving  the  best \nprofile for  a  new  perturbed schedule  that is  similar  to one  which  has  already  been \nlearned.  The degree of similarity required for  the generalization remains a  topic for \nfurther study. \n\n\f808 \n\nKeymeulen and de Gerlache \n\n5  Conclusion \n\nIn  conclusion, we  have shown  that the rescheduling  problem of an airline crew  pool \ncan  be  stated as  a  decision  making  problem,  namely  the  identification  of the  best \npotential substitute pilot.  We have stressed the importance of the codification of the \ninformation used by  the expert to evaluate the best candidate.  We have applied the \nneural network  learning  approach  to help  the rescheduler team in  the rescheduling \nprocess  by  using  the  experience  of already  solved  rescheduling  problems.  By  a \nmathematical  analysis  we  have  proven  the  efficiency  of  the  dual  neural  network \narchitecture.  The mathematical  analysis  permits  also  to improve  the  convergence \nof the  network.  Finally  we  have  illustrated  the  method  on  rescheduling  problems \nfor  the Sabena Belgian  Airline  company. \n\nAcknowledgments \n\nWe  thank  the  Scheduling  and  Rescheduling  team  of Mr.  Verworst  at  Sabena for \ntheir valuable information given all along this study; Professors Steels and D'Hondt \nfrom the VUB and Professors Pastijn, Leysen and Declerck from  the Military Royal \nAcademy  who  supported  this  research;  Mr.  Horner  and  Mr.  Pau  from  the  Digital \nEurope organization  for  their  funding.  We  specially  thank  Mr.  Decuyper  and  Mr. \nde  Gerlache  for  their  advices  and attentive reading. \n\nReferences \n\nIn  Proceedings  of Fourth  International \n\nH.  Braun, J.  Faulner &  V.  Uilrich.  (1991)  Learning strategies for  solving the prob(cid:173)\nlem  of  planning  using  backpropagation. \nConference  on  Neural  Networks  and their Applications,  671-685.  Nimes,  France. \nJ.  Bromley,  I. Guyon, Y . Lecun,  E. Sackinger,  R.  Shah .  (1994).  Signature verifica(cid:173)\ntion  using a siamese delay  neural network.  In  J.  Cowan, G.  Tesauro & J. Alspector \n(eds.),  Advances  in  Neural  Information  Processing  Systems  1.  San  Mateo,  CA: \nMorgan  Kaufmann. \nM.  de Gerlache & D.  Keymeulen.  (1993) A neural network learning strategy adapted \nfor  a  rescheduling  problem.  In  Proceedings  of Fourth  International  Conference  on \nNeural  Networks  and  their Applications,  33-42 .  Nimes,  France. \nD.  Rumelhart  &  J.  McClelland. \nrations  in  the  Microstructure  of Cognition  I  [1  II.  Cambridge,  MA:  MIT  Press. \n\n(1986)  Parallel  Distributed  Processing:  Explo(cid:173)\n\nL.  Steels.  (1990)  Components of expertise.  AI Maga.zine,  11(2):29-49 . \n\n(1989)  Connectionist  learning  of  expert  preferences  by  comparison \nIn  D.  S.  Touretzky  (ed.),  Advances  in  Neural  Information  Processing \n\nG.  Tesauro. \ntraining. \nSystems  1,  99-106.  San  Mateo,  CA:  Morgan  Kaufmann. \nG.  Tesauro  &  T.J.  Sejnowski. \nbackgammon.  Artificial Intelligence,  39:357-390. \n\n(1989)  A  parallel  network  that  learns  to  play \n\n\f", "award": [], "sourceid": 871, "authors": [{"given_name": "Didier", "family_name": "Keymeulen", "institution": null}, {"given_name": "Martine", "family_name": "de Gerlache", "institution": null}]}