{"title": "Comparison Training for a Rescheduling Problem in Neural Networks", "book": "Advances in Neural Information Processing Systems", "page_first": 801, "page_last": 808, "abstract": null, "full_text": "Comparisoll Training for a Resclleduling \n\nProblem ill Neural Networks \n\nDidier Keymeulen \n\nArtificial Intelligence Laboratory \n\nVrije Universiteit Brussel \nPleinlaan 2, 1050 Brussels \n\nBelgium \n\nMartine de Gerlache \n\nProg Laboratory \n\nVrije Universiteit Brussel \nPleinlaan 2, 1050 Brussels \n\nBelgium \n\nAbstract \n\nAirline companies usually schedule their flights and crews well in \nadvance to optimize their crew pools activities. Many events such \nas flight delays or the absence of a member require the crew pool \nrescheduling team to change the initial schedule (rescheduling). In \nthis paper, we show that the neural network comparison paradigm \napplied to the backgammon game by Tesauro (Tesauro and Se(cid:173)\njnowski, 1989) can also be applied to the rescheduling problem of \nan aircrew pool. \nIndeed both problems correspond to choosing \nthe best solut.ion from a set of possible ones without ranking them \n(called here best choice problem). The paper explains from a math(cid:173)\nematical point of view the architecture and the learning strategy of \nthe backpropagation neural network used for the best choice prob(cid:173)\nlem. We also show how the learning phase of the network can be \naccelerated. Finally we apply the neural network model to some \nreal rescheduling problems for the Belgian Airline (Sabena). \n\n1 \n\nIntroduction \n\nDue to merges, reorganizations and the need for cost reduction, airline companies \nneed to improve the efficiency of their manpower by optimizing the activities of \ntheir crew pools as much as possible. A st.andard scheduling of flights and crews is \nusually made well in advance but many events, such as flight delays or the absence \nof a crew member make many schedule cha.nges (rescheduling) necessary. \n\n801 \n\n\f802 \n\nKeymeulen and de Gerlache \n\nEach day, the CPR 1 team of an airline company has to deal with these pertur(cid:173)\nbations. The problem is to provide the best answer to these regularly occurring \nperturbations and to limit their impact on the general schedule. Its solution is hard \nto find and usually the CPR team calls on full reserve crews. An efficient reschedul(cid:173)\ning tool taking into account the experiences of the CPR team could substantially \nreduce the costs involved in rescheduling notably by limit.ing the use of a reserve \ncrew. \n\nThe paper is organized as follow. In the second section we describe the rescheduling \ntask. In the third section we argue for the use of a neural network for the reschedul(cid:173)\ning task and we apply an adequate architecture for such a network. Finally in \nthe last section, we present results of experiments with schedules based on actual \nschedules used by Sabena. \n\n2 Rescheduling for an Airline Crew Pool \n\nWhen a pilot is unavailable for a flight it becomes necessary to replace him, e.g. \nto reschedule the crew. The rescheduling starts from a list of potential substitute \npilots (PSP) given by a scheduling program based generally on operation research or \nexpert syst.em technology (Steels, 1990). The PSP list obtained respects legislation \nand security rules fixing for example t.he number of flying hours per month, the \nmaximum number of consecutive working hour and the number of training hours \nper year and t.heir schedule. From the PSP list, the CPR team selects the best \ncandidat.es taking into account t.he schedule stability and equity. The schedule \nstability requires that possible perturbations of the schedule can be dealt with with \nonly a minimal rescheduling effort. This criterion ensures work stability t.o the crew \nmembers and has an important influence on their social behavior. The schedule \nequity ensures the equal dist.ribution of the work and payment among the crew \nmembers during the schedule period. \n\nOne may think to solve this rescheduling problem in t.he same way as the scheduling \nproblem itself using software t.ools based on operational research or expert system \napproach. But t.his is inefficient. for t.wo reasons, first., the scheduling issued from a \nscheduling system and its adapt.at.ion t.o obt.ain an acceptable schedule takes days. \nSecond this system does not t.ake into account the previous schedule. It follows \nthat the updat.ed one may differ significantly from the previous one after each \nperturbation. This is unaccept.able from a pilot's point of view. Hence a specific \nprocedure for rescheduling is necessary. \n\n3 Neural Network Approach \n\nThe problem of reassigning a new crew member to replace a missing member can \nbe seen as the problem of finding the best pilot in a pool of potential substitute \npilots (PSP), called the best choice problem. \n\nTo solve the best choice problem, we choose the neural network approach for two \nreasons. First the rules llsed by the expert. are not well defined: to find the best PSP, \n\nlCrew Pool Rescheduler \n\n\fComparison Training for a Rescheduling Problem in Neural Networks \n\n803 \n\nthe expert associates implicit.ly a score value to each profile. The learning approach \nis precisely well suited to integrate, in a short period of time, t.he expert knowledge \ngiven in an implicit form. Second, t.he neural network approach was applied with \nsuccess to board-games e.g. the Backgammon game described by Tesauro (Tesauro \nand Sejnowski, 1989) and the Nine l\\llen's Morris game described by Braun (Braun \nand al., 1991). These two games are also exa.mples of best choice problem where \nthe player chooses the best move from a set of possible ones. \n\n3.1 Profile of a Potential Substitute Pilot \n\nTo be able to use the neural network approach we have to identify the main fea(cid:173)\ntures of the potential substitute pilot and to codify them in terms of rating values \n(de Gerlache and Keymeulen, 1993). We based our coding scheme on the way the \nexpert solves a rescheduling problem. He ident.ifies the relevant parameters associ(cid:173)\nated with the PSP and the perturbed schedule. These parameters give three types \nof information. A first type describes the previous, present and future occupation \nof the PSP. The second t.ype represents information not in the schedule such as \nthe human relationship fadars. The assocjat.ed values of t.hese two t.ypes of pa(cid:173)\nrameters differ for f'ach PSP. The last t.ype of paramet.ers describes the context \nof the rescheduling, namely t.he characteristics of t.he schedule . This last type of \nparameters are the same for all the PSP. All t.hese paramet.ers form the profile of \na PSP associated to a perturbed schedule. At each rescheduling problem corre(cid:173)\nsponds one perturbed schedule j and a group of 11 PSpi to which we associate a \n\nProjile~ = (PSpi, PertU1\u00b7berLSchedulej) . Implicitly, the expert associates a rat-\n\ning value between 0 and 1 to each parameter of the P1'ojile; based on respectively \nits little or important impact on the result.ing schedule if the P S pi was chosen. \nThe rating value reflects the relative importance of the parameters on the stability \nand the equity of the resulting schf'dnle obt.ained after the pilots substitution. \n\n3.2 Dual Neural Network \n\nIt would have been possible to get more information from the expert than only the \nbest profile. One of the possibilities is to ask him to score every profile associated \nwith a perturbed planning. From this associat.ion we could immediately construct \na scoring function which couples each profile with a specific value, namely its score. \nAnother possibility is to ask the expert to rank all profiles associated with a per(cid:173)\nturbed schedule. The corresponding ranking function couples each profile with a \nvalue such that the values associat.ed with the profiles of the same perturbed sched(cid:173)\nule order the profiles according t.o t.heir rank. The decision making process used by \nthe rescheduler team for the aircrew rescheduling problem does not consist in the \nevaluation of a scoring or ranking function . Indeed only the knowledge of the best \nprofile is useful for the rescheduling process. \n\nFrom a neura.l net.work architectural point of view, because the ranking problem is a \ngeneralization of the best choice problem, a same neural net.work architecture can be \nused. But the difference between the best choice problem and t.he scoring problem \nis such that two different neural network architectures are associated to them. As \nwe show in this section, although a backpropagatian network is sufficient to learn a \nscoring function, its architecture, its learning and its retrieval procedures must be \n\n\f804 \n\nKeymeulen and de Gerlache \n\nadapted to learn the best profile. Through a mathematical formulation of the best \nchoice problem, we show that the comparison paradigm of Tesauro (Tesauro, 1989) \nis suited to the best choice problem and we suggest how to improve the learning \nconvergence. \n\n3.2.1 Comparing Function \n\nFor the best choice problem the expert gives the best profile Projilefest associated \nwith the perturbed schedule j and that for m pert.urbed schedules. The problem \nconsists then to learn the mapping of the m * n profiles associated with the m \nperturbed schedules into the m best profiles, one for each pert.urbed schedule. One \nway to represent this association is through a comparing function. This function \nhas as input a profile, represented by a vector xj, and returns a single value. When \na set of profiles associated with a perturbed schedule are evaluated by the function, \nit returns the lowest value for the best profile. This comparing function integrates \nthe information given by the expert and is sufficient to reschedule any perturbed \nschedule solved in the past by the expert. Formally it is defined by: \n\nComp(J.1>e) = C(Projile)) \n\n(1) \n\nC \n\nomparej \n\nBest C \n\n< ompcl1>Cj \n\n,. {V)' \n\nVi=fBest with \n\nwith)' = 1, ... ,111. \ni=l, ... ,n \n\nThe value of Comp(J.1>e) are not known a priori and have only a meaning when they \nare compared to the value Comp(J.1>ef est of the comparing function for the best \nprofile. \n\n3.2.2 Geometrical Interpretation \n\nTo illustrate the difference between the neural network learning of a scoring function \nand a comparing function, we propose a geometrical interpretation in the case of \na linear network having as input vect.ors (profiles) XJ, ... ,XJ, ... ,Xp associated \nwith a perturbed schedule j. \n\nThe learning of a scoring function which associat.es a score Score; with each input \nvector xj consists in finding a hyperplane in the input vector space which is tangent \nto the circles of cent.er xf and radius SC01>e{ (Fig. 1). On the contrary the learning \nof a comparing function consists t.o obt.ain t.he equation of an hyperplane such that \nthe end-point of the vector Xfest is nearer the hyperplane than the end-points of \nthe other input vectors XJ associated with the same perturbed schedule j (Fig. 1). \n\n3.2.3 Learning \n\nWe use a neural network approach to build the comparing function and the mean \nsquared error as a measure of the quality of t.he approximation. The comparing \nfunction is approximated by a non-linear function: C(P1>ojile;) = N\u00a3(W,Xj) \nwhere W is the weight. vector of the neural network (e.g backpropagat.ion network). \nThe problem of finding C which has the property of (1) is equivalent to finding the \nfunction C that minimizes the following error function (Braun and al., 1991) where \n* is the sigmoid function : \n\n\fComparison Training for a Rescheduling Problem in Neural Networks \n\n805 \n\n... --_ .. _(cid:173)\n\n,--\n\nx,, \n\n\"\"\"\", , , \n, \n\\ , , , , \nI , , , , , , , \n\nI \n\n,.,1' \n\n-(cid:173)\n\n.,' \n\nW( Wl ,w2'''')' .. ,WL) \nwlh \n\n.w. - 1 \n\nx..,,, \n\nI \n\" \n', ....... --,' \n\nFigure 1: Geometrical Interpretation of the learning of a Scoring Function \n(Rigth) and a Comparing Function (Left) \n\nn \n\nI: \ni = 1 \ni -::f Best \n\n(2) \n\nTo obtain t.he weight vector which mll1UTIlzes the error funct.ion (2), we use the \nproperty that t.he -gr~ld\u00a3~(W) point.s in the direct.ion in which the error function \nwill decrease at the fastest possible rate. To update t.he weight we have thus to \ncalculate the partial derivative of (2) with each components of the weight vector \nltV: it is made of a product of three factors. The evaluation of the first two factors \n(the sigmoid and the derivative of the sigmoid) is immediate. The third factor is \nthe partial derivative of the non-linear function N \u00a3, which is generally calculated \nby using the generalized delta rule learning law (Rumelhart. and McClelland, 1986), \n\nUnlike the linear associator network, for the backpropagation network, the error \nfunction (2) is not equivalent to the error function where the difference Xl e3t - X; \nis associated with the input vector of the backpropagation network because: \n\n(3) \n\nBy consequence to calculate t.he three factors of the partial derivative of (2), we \nhave to introduce separately at the bottom of the network t.he input vector of the \nbest profile X !e3t and the input vector of a less good profile XJ. Then we have to \nmemorize theIl' partial contribution at each node of the network and multiply their \ncontributions before updating the weight . Using this way to evaluate the derivative \nof (2) and to update t.he weight, the simplicity of the generalized delta rule learning \nlaw has disappeared . \n\n\f806 \n\nKeymeulen and de Gerlache \n\n3.2.4 Architecture \n\nTesauro (Tesauro and Sejnowski, 1989) proposes an architecture, t.hat we call dual \nneural network, and a learning procedure such that the simplicity of the generalized \ndelta rule learning law can still be used (Fig. 2). The same kind of architecture, \ncalled siamese network, was recently used by Bromley for the signature verification \n(Bromley and al., 1994). The dual neural network architecture and the learning \nstrategy are justified mathematically at one hand by the decomposition of the partial \nderivative of the error function (2) in a sum of two terms and at the other hand by \nthe asymmetry property of the sigmoid and its derivative. \n\nThe architecture of the dual neural network consists to duplicate the multi-layer \nnetwork approximating the comparing function (1) and to connect the output of \nboth to a unique output node through a positive unit weight for the left network and \nnegative unit weight. for the right network. During the learning a couple of profiles \nis presented to the dual neural network: a best profile X f e3t and a less good profile \nX!. The desired value at the output node of the dual neural network is 0 when the \nleft network has for input the best profile and the right network has for input a less \ngood profile and 1 when these profiles are permuted. During the recall we work only \nwith one of the two multi-layer networks, suppose the left one (the choice is of no \nimportance because they are exactly the same). The profiles JY~ associated with a \nperturbed schedule j are presented at the input of the left. multi-layer network. The \nbest profile is the one having the lowest value at the output of the left multi-layer \nnetwork. \n\nThrough this mathematical formulation we can use the suggestion of Braun to \nimprove the learning convergence (Braun and al., 1991). They propose to replace \nthe positive and negative unit weight het.ween the output node of the multi-layer \nnetworks and the output. node of the dual neural network by respect.ively a weight \nvalue equal to V for the left net.work and - V for the right. network. They modify the \nvalue of V by applying the generalized delt.a rule which has no significant impact on \nthe learning convergence. By manually increasing the factor V during the learning \nprocedure, we improve considerably the learning convergence due to its asymmetric \nimpact on the derivative of \u00a3**(W) with W: the modification of the weight vector \nis greater for couples not yet learned than for couples already learned. \n\n4 Results \n\nThe experiments show the abilit.y of our model to help the CPR team of the Sabena \nBelgian Airline company to choose the best profile in a group of PSPs based on \nthe learned expertise of the team. To codify the profile we identify 15 relevant \nparameters. They constitute the input of our neural network. The training data \nset was obtained by analyzing the CPR team at work during 15 days from which \nwe retain our training and test perturbed schedules. \n\nWe consider that the network has learned when the comparing value of the best \nprofile is less than the comparing value of t.he other profiles and that for all training \nperturbed schedules. At that time \u00a3cJ>(W) is less t.han .5 for every couple of profiles. \nThe left graph of Figure 3 shows t.he evolution of t.he mean error over t.he couples \n\n\fComparison Training for a Rescheduling Problem in Neural Networks \n\n807 \n\nDual Neural Network \n\n\u00b71 \n\nC_ .. n \n\nBelt. \n\n.1Ir.(~.l \n\n) \nBeltoJ \n\nZMulI.La,'\"\" \n\n-'-od ..... \n\nNouol \nNoI\",ort.o \n\nC_,.n \n'J \n\n.11 r.(~. t .) \n\n,~ \n\n10.6lo.~ 10,910\", 10.110.710.11 XB .... I I63IO.2!O-.1 b.61 0.21 0.91 0.21 X2\u20221 \n10.310.210.11 0.61 0.21 0.91 0.21 X 2\u20221 10,616316.91 631 0.11 6.71 0.21 X \n\nB .... I \n\n-+ \n(X \n\n-+ \n.X \ntoJ \n\nBut~ \n\n) \n\n\u2022 \n\nwdb XIJ = X But.l \n\nit.t ) 10.610.710.1/ 0.11 0.711.01 0.21 XB ..... lo. 1Io.9Io.7! 0.910.110.11 0.&1 X I .. \n,~ B.... \n10.610.710.1( 0.11 0.711.0 I 0.11 XB ..... \n\n10.11 0,91 0.71 0.91 0.11 0.11 0,61 X I.. \nWllb 'S .. = X Be ... \n\nFigure 2: The training of a dual neural network. \n\nduring the training . The right graph shows the improvement of the convergence \nwhen the weight V is increased regularly during the training process. \n\nOJ \n\nOA \n\n0.1 \n\nlnclasilll V \n\nor the Dull NaIl'll Ndworll \n\no.s \n\n0 .\u2022 \n\nOJ \n\nM~--~~~----------------------\n\n02 ~--~~------------------------\n\n0.1 \n\n01 \n\nliIl) \n\nliIO \n\n111(1) \n\nIlO) \n\nNwnbcr \nof'lhilitg \n\nNunt.crci \n~~~~~~~~~~~~~~~~ nW~ \n\nFigure 3: Convergence of the dual neural network architecture. \n\nThe network does not converge when we introduce contradictory decisions in our \ntraining set. It is possible to resolve them by adding new context parameters in the \ncoding scheme of the profile. \n\nAfter learning, our network shows generalization capacity by retrieving the best \nprofile for a new perturbed schedule that is similar to one which has already been \nlearned. The degree of similarity required for the generalization remains a topic for \nfurther study. \n\n\f808 \n\nKeymeulen and de Gerlache \n\n5 Conclusion \n\nIn conclusion, we have shown that the rescheduling problem of an airline crew pool \ncan be stated as a decision making problem, namely the identification of the best \npotential substitute pilot. We have stressed the importance of the codification of the \ninformation used by the expert to evaluate the best candidate. We have applied the \nneural network learning approach to help the rescheduler team in the rescheduling \nprocess by using the experience of already solved rescheduling problems. By a \nmathematical analysis we have proven the efficiency of the dual neural network \narchitecture. The mathematical analysis permits also to improve the convergence \nof the network. Finally we have illustrated the method on rescheduling problems \nfor the Sabena Belgian Airline company. \n\nAcknowledgments \n\nWe thank the Scheduling and Rescheduling team of Mr. Verworst at Sabena for \ntheir valuable information given all along this study; Professors Steels and D'Hondt \nfrom the VUB and Professors Pastijn, Leysen and Declerck from the Military Royal \nAcademy who supported this research; Mr. Horner and Mr. Pau from the Digital \nEurope organization for their funding. We specially thank Mr. Decuyper and Mr. \nde Gerlache for their advices and attentive reading. \n\nReferences \n\nIn Proceedings of Fourth International \n\nH. Braun, J. Faulner & V. Uilrich. (1991) Learning strategies for solving the prob(cid:173)\nlem of planning using backpropagation. \nConference on Neural Networks and their Applications, 671-685. Nimes, France. \nJ. Bromley, I. Guyon, Y . Lecun, E. Sackinger, R. Shah . (1994). Signature verifica(cid:173)\ntion using a siamese delay neural network. In J. Cowan, G. Tesauro & J. Alspector \n(eds.), Advances in Neural Information Processing Systems 1. San Mateo, CA: \nMorgan Kaufmann. \nM. de Gerlache & D. Keymeulen. (1993) A neural network learning strategy adapted \nfor a rescheduling problem. In Proceedings of Fourth International Conference on \nNeural Networks and their Applications, 33-42 . Nimes, France. \nD. Rumelhart & J. McClelland. \nrations in the Microstructure of Cognition I [1 II. Cambridge, MA: MIT Press. \n\n(1986) Parallel Distributed Processing: Explo(cid:173)\n\nL. Steels. (1990) Components of expertise. AI Maga.zine, 11(2):29-49 . \n\n(1989) Connectionist learning of expert preferences by comparison \nIn D. S. Touretzky (ed.), Advances in Neural Information Processing \n\nG. Tesauro. \ntraining. \nSystems 1, 99-106. San Mateo, CA: Morgan Kaufmann. \nG. Tesauro & T.J. Sejnowski. \nbackgammon. Artificial Intelligence, 39:357-390. \n\n(1989) A parallel network that learns to play \n\n\f", "award": [], "sourceid": 871, "authors": [{"given_name": "Didier", "family_name": "Keymeulen", "institution": null}, {"given_name": "Martine", "family_name": "de Gerlache", "institution": null}]}*