{"title": "The Parameterized Complexity of Cascading Portfolio Scheduling", "book": "Advances in Neural Information Processing Systems", "page_first": 7668, "page_last": 7678, "abstract": "Cascading portfolio scheduling is a static algorithm selection strategy which uses a sample of test instances to compute an optimal ordering (a cascading schedule) of a portfolio of available algorithms. The algorithms are then applied to each future instance according to this cascading schedule, until some algorithm in the schedule succeeds. Cascading algorithm scheduling has proven to be effective in several applications, including QBF solving and the generation of ImageNet classification models.\n\nIt is known that the computation of an optimal cascading schedule in\nthe offline phase is NP-hard. In this paper we study the parameterized complexity of this problem and establish its fixed-parameter tractability by utilizing structural properties of the success relation between algorithms and test instances.\n\nOur findings are significant as they reveal that in spite of the intractability of the problem in its general form, one can indeed exploit sparseness or density of the success relation to obtain non-trivial runtime guarantees for finding an optimal cascading schedule.", "full_text": "The Parameterized Complexity of\nCascading Portfolio Scheduling\n\nEduard Eiben\nRoyal Holloway\n\nUniversity of London\n\nDepartment of CS\n\nUK\n\nRobert Ganian\n\nTU Wien\n\nAlgorithms and\n\nComplexity Group\n\nAustria\n\nIyad Kanj\n\nDePaul University\n\nSchool of Computing\n\nChicago\n\nUSA\n\nStefan Szeider\n\nTU Wien\n\nAlgorithms and\n\nComplexity Group\n\nAustria\n\nAbstract\n\nCascading portfolio scheduling is a static algorithm selection strategy which uses a\nsample of test instances to compute an optimal ordering (a cascading schedule) of\na portfolio of available algorithms. The algorithms are then applied to each future\ninstance according to this cascading schedule, until some algorithm in the schedule\nsucceeds. Cascading scheduling has proven to be effective in several applications,\nincluding QBF solving and generation of ImageNet classi\ufb01cation models.\nIt is known that the computation of an optimal cascading schedule in the of\ufb02ine\nphase is NP-hard. In this paper we study the parameterized complexity of this\nproblem and establish its \ufb01xed-parameter tractability by utilizing structural prop-\nerties of the success relation between algorithms and test instances. Our \ufb01ndings\nare signi\ufb01cant as they reveal that in spite of the intractability of the problem in its\ngeneral form, one can indeed exploit sparseness or density of the success relation\nto obtain non-trivial runtime guarantees for \ufb01nding an optimal cascading schedule.\n\nIntroduction\n\n1\nWhen dealing with hard computational problems, one often has access to a portfolio of different\nalgorithms that can be applied to solve the given problem, with each of the algorithms having\ncomplementary strengths. There are various ways of how this performance complementarity can be\nexploited. Algorithm selection, a line of research initiated by Rice [19], studies various approaches\none can use to select algorithms from the portfolio. Algorithm selection has proven to be an extremely\npowerful tool with many success stories in Propositional Satis\ufb01ability, Constraint Satisfaction,\nPlanning, QBF Solving, Machine Learning and other domains [12, 13, 14, 20]. A common approach\nto algorithm selection is per-instance-based algorithm selection, where an algorithm is chosen for\neach instance independently, based on some features of the instance (see, e.g., [15, 10]). However,\nsometimes information about the individual instances is not available or dif\ufb01cult to use. Then, one\ncan instead make use of information about the distribution of the set of instances, e.g., in terms of\na representative sample of instances which can be used as a training set. In such cases, one can\ncompute in an of\ufb02ine phase a suitable linear ordering of the algorithms, optimizing the ordering for\nthe training set of instances. This ordering is then applied uniformly to any given problem instance in\nan online fashion\u2014in particular, if the \ufb01rst algorithm in our ordering fails to solve a given instance\n(due to timeout, memory over\ufb02ow, or due to not reaching a desired accuracy), then the second\nalgorithm is called, and this continues until we solve the instance. Such a static algorithm selection,\n\u201ccascading portfolio scheduling\u201d, is simpler to implement than per-instance selection methods and\ncan be very effective [22]. One prominent recent application of cascading portfolio scheduling lies in\nstate-of-the-art ImageNet classi\ufb01cation models, where it resulted in a signi\ufb01cant speedup by reducing\nthe number of \ufb02oating-point operations [23]. Cascading portfolio scheduling is also related to online\nportfolio scheduling [11, 16].\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fIn this paper we address the fundamental problem of \ufb01nding an optimal cascading schedule for a\ngiven portfolio A of algorithms with respect to a given training set T of instances. In particular,\nfor the problem CASCADING PORTFOLIO SCHEDULING (or CPS for short) that we consider, we\nare given m algorithms, n test instances, and a cost mapping cost, where cost(\u03b1, t) denotes the cost\nof running algorithm \u03b1 on test instance t, and a success relation S where (\u03b1, t) \u2208 S means that\nalgorithm \u03b1 succeeds on test instance t. As the cost mapping and the success relation are de\ufb01ned\nindependently, this setting is very general and entails different scenarios.\nScenario 1 Each algorithm is run until a globally set timeout C is reached. If the algorithm \u03b1 solves\ntest instance t in time c \u2264 C then cost(\u03b1, t) = c and (\u03b1, t) \u2208 S; otherwise we have cost(\u03b1, t) = C\nand (\u03b1, t) /\u2208 S.\nScenario 2 Algorithm \u03b1 solves a test instance t in time c and outputs an accuracy estimate r for its\nsolution. r is then compared with a globally set accuracy threshold R. If r \u2265 R then (\u03b1, t) \u2208 S,\notherwise (\u03b1, t) /\u2208 S; in any case cost(\u03b1, t) = c. Such a strategy has been used for prediction\nmodel generation [23].\n\nScenario 3 All the algorithms are \ufb01rst run with a short timeout and if the test instance has not been\nsolved after this, algorithms are run again without a timeout (a similar strategy has been used for\nQBF solving [18]). Such a strategy can be instantiated to our setting by adding two copies of each\nalgorithm to the portfolio, one with a short timeout and one without a timeout.\n\nContribution. We establish the \ufb01xed-parameter tractability1 of computing an optimal cascading\nschedule by utilizing structural properties of the success relation. We look at the success relation in\nterms of a Boolean matrix, the evaluation matrix, where each row corresponds to a test instance and\neach column corresponds to an algorithm. A cell contains the entry 1 iff the corresponding algorithm\nsucceeds on the corresponding test. We show that if this matrix is either very sparse or very dense,\nthen the computation of an optimal schedule is tractable. More speci\ufb01cally, we establish the following\nresults, which we describe by writing CPS[parm] for CASCADING PORTFOLIO SCHEDULING\nparameterized by parameter parm.\nFirst we consider the algorithm failure degree which is the largest number of tests a single algorithm\nfails on, and the test failure degree which is the largest number of algorithms that fail on a single\ntest (these two parameters can also be seen as the largest number of 0\u2019s that appear in a row and the\nlargest number of 0\u2019s that appear in a column of the matrix, respectively).\n(1) CPS[algorithm failure degree] and CPS[test failure degree] are \ufb01xed-parameter tractable (Theo-\n\nrems 4 and 5).\n\nIt is natural to consider also the dual parameters algorithm success degree and test success degree.\nHowever, it follows from known results that CPS is already NP-hard if both of these parameters\nare bounded by a constant (Proposition 6). Hence, our results exhibit a certain asymmetry between\nfailure and success degrees.\nWe then consider more sophisticated parameters that capture the sparsity or density of the evaluation\nmatrix. The failure cover number is the smallest number of rows and columns in the evaluation\nmatrix needed to cover all the 0\u2019s in the matrix; similarly, the success cover number is the smallest\nnumber of rows and columns needed to cover all the 1\u2019s. In fact, both parameters can be computed in\npolynomial time using bipartite vertex cover algorithms [7].\n(2) CPS[failure cover number] and CPS[success cover number] are \ufb01xed-parameter tractable\n\n(Corollary 8 and Theorem 16).\n\nThese results are signi\ufb01cant as they indicate that CASCADING PORTFOLIO SCHEDULING can be\nsolved ef\ufb01ciently as long as the evaluation matrix is suf\ufb01ciently sparse or dense. Our result for\nCPS[failure cover number] in fact also shows \ufb01xed-parameter tractability of the problem for an even\nmore general parameter than success cover number: the treewidth [21] of the bipartite graph between\nthe algorithms and tests, where edges join success pairs. This is our most technical contribution and\nreveals how a fundamental graphical parameter [see, e.g., 8] can be utilized for algorithm scheduling.\nAnother natural variant of the problem, CPSopt[length], arises by adding an upper bound (cid:96) on the\nlength, i.e., cardinality, of the computed schedule, and asking for a schedule of length \u2264 (cid:96) of minimum\ncost. We obtain a complexity classi\ufb01cation of the problem under this parameterization as well.\n\n1Fixed-parameter tractability is a relaxation of polynomial tractability; de\ufb01nitions are provided in Section 2.\n\n2\n\n\f(3) CPS[length] can be solved in polynomial time for each \ufb01xed bound (cid:96), but is not \ufb01xed-parameter\n\ntractable parameterized by (cid:96) subject to established complexity assumptions.\n\nAn overview of our results is provided in Table 1.\n\nParameter\nAlgorithm failure degree\nTest failure degree\nAlgorithm and test success degree\nFailure cover number and failure treewidth\nSuccess cover number\nLength\n\nComplexity\nFPT\nFPT\nNP-hard (for constant parameters)\nFPT\nFPT\nin XP and W[2]-hard\n\nReference\nProposition 4\nProposition 5\nProposition 6\nTheorem 7\nTheorem 16\nProposition 3\n\nTable 1: An overview of the complexity results presented in this paper.\n\n2 Preliminaries\nProblem De\ufb01nition. An instance of the CASCADING PORTFOLIO SCHEDULING problem is a\ntuple (A, T, cost, S) comprising:\n\u2022 a set A of m algorithms,\n\u2022 a set T of n tests,\n\u2022 a cost mapping cost : (A \u00d7 T ) \u2192 N, and\n\u2022 a success relation S \u2286 A \u00d7 T .\nLet \u03c4 be a totally ordered subset of A; we call such a set a schedule. The length of a schedule is\nits cardinality. We say that \u03c4 is valid if for each test t there exists an algorithm \u03b1 \u2208 \u03c4 such that\n(\u03b1, t) \u2208 S. Throughout the paper, we will assume that there exists a valid schedule for our considered\ninstances\u2014or, equivalently, that each test is solved by at least one algorithm.\n\nThe processing cost of a test t for a valid schedule \u03c4 = (\u03b11, . . . , \u03b1q) is de\ufb01ned as(cid:80)j\n\ni=1 cost(\u03b1i, t),\nwhere j is the \ufb01rst algorithm in \u03c4 such that (\u03b1j, t) \u2208 S. The cost of a valid schedule \u03c4, denoted\ncost(\u03c4 ), is the sum of the processing costs of all tests in T for \u03c4. The aim in CASCADING PORTFOLIO\nSCHEDULING is to \ufb01nd a valid schedule \u03c4 of minimum cost.\nParameterized Complexity.\nIn parameterized algorithmics [6, 4, 3, 9] the complexity of a problem\nis studied not only with respect to the input size n but also a parameter k \u2208 N. The most favorable\ncomplexity class in this setting is FPT (\ufb01xed-parameter tractable) which contains all problems that\ncan be solved by an algorithm running in time f (k) \u00b7 nO(1), where f is a computable function.\nAlgorithms running in this time are called \ufb01xed-parameter algorithms. We will also make use of the\ncomplexity classes W[2] and XP, where W[2] \u2286 XP. Problems complete for W[2] are are widely\nbelieved to not be in FPT. The class XP contains problems that are solvable in time O(nf (k)),\nwhere f is a computable function; in other words, problems in XP are polynomial-time solvable\nwhen the parameter is bounded by a constant. To obtain our lower bound results, we will need the\nnotion of a parameterized reduction, referred to as FPT-reduction, which is in many ways analogous\nto the standard polynomial-time reductions; the distinction is that a parameterized reduction runs in\ntime f (k) \u00b7 nO(1) for some computable function f, and provides upper bounds on the parameter size\nin the resulting instance [4, 3, 6, 17].\nWe write O\u2217(f (k)) to denote a function of the form f (k) \u00b7 nO(1), where n is the input length and k\nis the parameter.\nProblem Parameters. CASCADING PORTFOLIO SCHEDULING is known to be NP-hard [23], and\nour aim in this paper will be to circumvent this by obtaining parameters that exploit the \ufb01ne-grained\nstructure in relevant problem instances. We note that we explicitly aim for results which allow for\narbitrary cost mappings, since these are expected to consist of large (and often disorderly) numbers in\nreal-life settings. Instead, we will consider parameters that restrict structural properties of the \u201cbinary\u201d\nsuccess relation. To visualize this success relation, it will be useful to view an instance I as an m \u00d7 n\nmatrix MI where MI[i, j] = 1 if (\u03b1i, tj) \u2208 S (i.e. if the j-th test succeeds on the i-th algorithm, for\nsome \ufb01xed ordering of algorithms and tests), and MI[i, j] = 0 otherwise.\n\n3\n\n\fFigure 1: An instance with 4 algorithms and 5 tests in the setting where (exact) algorithms are executed\nwith a global timeout of 7, as discussed in Scenario 1. On the left is the matrix MI representing the\nsuccess relation. The failure covering number is 3, as witnessed by the highlighted two rows and one\ncolumn. The matrix CI on the right represents the cost relation, with CI[i, j] = cost[\u03b1i, tj]. The\ninstance I depicted here has a single solution, notably (\u03b11, \u03b13).\n\nThe two most natural parameters to consider are m and n, and these correspond to the number of rows\nand columns in MI, respectively. Unfortunately, these two parameters are also fairly restrictive\u2014it\nis unlikely that instances of interest will have a very small number of algorithms or test instances.\nAnother option would be to use the maximum number of times an algorithm (or test) can fail (or\nsucceed) as a parameter. In particular, the algorithm success (or failure) degree is the maximum\nnumber of 1\u2019s (or 0\u2019s, respectively) occurring in any row in MI. Similarly, we let the test success (or\nfailure) degree be the maximum number of 1\u2019s (or 0\u2019s, respectively) occurring in any column in MI.\nInstances where these parameters are small correspond to cases where \u201calmost everything\u201d either\nfails or succeeds.\nA more advanced parameter that can be extracted from MI is the covering number, which intuitively\ncaptures the minimum number of rows and columns that are needed to \u201ccover\u201d all successes (or\nfailures) in the matrix. More formally, we say that an entry MI[i, j] is covered by row i and by\ncolumn j. Then the success (or failure) covering number is the minimum value of r + c such that\nthere exist r rows and c columns in MI with the property that each occurrence of 1 (or 0, respectively)\nin MI is covered by one of these rows or columns. Intuitively, an instance has success covering\nnumber s if there exist r algorithms and s \u2212 r tests such that these have a non-empty intersection\nwith every relation in S\u2014see Figure 1 for an example. We note that the covering number has been\nused as a structural parameter of matrices, notably in previous work on the MATRIX COMPLETION\nproblem [7], and that it is possible to compute r algorithms and c tests achieving a minimum covering\nnumber in polynomial time [7, Proposition 1]. We will denote the success covering number by covs\nand the failure covering number by covf .\n3 Results for Basic Parameters\nIn this section we consider the CASCADING PORTFOLIO SCHEDULING problem parameterized by\nthe number of algorithms (i.e., by m = |A|), by the number of tests (i.e., by n = |T|), and by the\nlength of the computed schedule.\nWe begin mapping the complexity of our problem with two initial propositions. Note that both\npropositions can also be obtained as corollaries of the more general Theorem 16, presented later. Still,\nwe consider it useful to present a short sketch of proof of Proposition 1, since it nicely introduces the\ncombinatorial techniques that will later be extended in the proof of Theorem 1.\nProposition 1. CPS[number of algorithms] is in FPT.\nProof Sketch. We reduce the problem to that of \ufb01nding a minimum-weight path in a directed acyclic\ngraph (DAG) D. We construct D as follows. We create a single source vertex s, and a single\ndestination vertex z in D. We de\ufb01ne L0 = {s}, Lm+1 = {z}, and apart from z, D contains m layers,\nL0, . . . , Lm, of vertices, where layer Li, for i \u2208 {0, . . . , m}, contains a vertex for each subset of A of\ncardinality i, with vertex s corresponding to the empty set. We connect each vertex that corresponds\nto a subset of A which is a valid portfolio to z. For each vertex u in layer Li, i \u2208 {0, . . . , m \u2212 1},\ncorresponding to a subset Su \u2282 A, and each vertex v \u2208 Li+1 corresponding to a subset Sv \u2286 A,\nwhere Sv = Su \u222a {\u03b1}, for \u03b1 \u2208 A, we add an edge (u, v) if there exists a test t \u2208 T such that (1)\n(\u03b1, t) \u2208 S and (2) there does not exist \u03b2 \u2208 Su such that (\u03b2, t) \u2208 S; in such case the weight of (u, v),\nwt(u, v), is de\ufb01ned as follows. Let T\u03b1 \u2286 T be the set of tests that cannot be solved by any algorithm\ncost(\u03b1, t). Informally speaking, the weight of (u, v) is the additional\ncost incurred by appending algorithm \u03b1 to any (partial) portfolio consisting of the algorithms in Su.\nThis completes the construction of D.\n\nin Su. Then wt(u, v) =(cid:80)\n\nt\u2208T\u03b1\n\n4\n\n\uf8eb\uf8ec\uf8ec\uf8ec\uf8ec\uf8ec\uf8ec\uf8ec\uf8ec\uf8ed11101001010101011101\uf8f6\uf8f7\uf8f7\uf8f7\uf8f7\uf8f7\uf8f7\uf8f7\uf8f7\uf8f8t1t2t3t4t5\u03b11\u03b12\u03b13\u03b14\uf8eb\uf8ec\uf8ec\uf8ec\uf8ec\uf8ec\uf8ec\uf8ec\uf8ec\uf8ed15273773757176725374\uf8f6\uf8f7\uf8f7\uf8f7\uf8f7\uf8f7\uf8f7\uf8f7\uf8f7\uf8f8MICI\fIt is not dif\ufb01cult to show that an optimal portfolio for A corresponds to a minimum-weight path from\ns to z, which can be computed in time O\u2217(2m).\nProposition 2. CPS[number of tests] is in FPT.\n\nTo formally capture the parameterization of the problem by the length (cid:96) of the computed schedule,\nwe need to slightly adjust its formal de\ufb01nition. Let CPSval[length] and CPSopt[length] denote the\nvariants of CASCADING PORTFOLIO SCHEDULING where for each problem instance we are also\ngiven an integer (cid:96) > 0 and only schedules up to length (cid:96) are considered ((cid:96) being the parameter).\nCPSval[length] is the decision problem that asks whether there exists a valid schedule of length \u2264 (cid:96),\nand CPSopt[length] asks to compute a valid schedule of length \u2264 (cid:96) of smallest cost or decide that no\nvalid schedule of length \u2264 (cid:96) exists. Both problems are parameterized by the length (cid:96).\nProposition 3. CPSopt[length] is in XP, but is unlikely to be in FPT since already CPSval[length] is\nW[2]-complete.\nProof Sketch. Membership of CPSopt[length] in XP is easy: We enumerate every ordered selection\nof at most (cid:96) algorithms from A (there are at most O((cid:96)!m(cid:96)) many) and if valid, we compute its cost,\nand keep track of a valid selection (if any) of minimum cost over all enumerations.\nTo prove the W[2]-hardness of CPSval[length], we give an FPT-reduction from the W[2]-complete\nproblem SET COVER [4]. The membership of CPSval[length] in W[2] follows from a straightforward\nreduction to SET COVER, which is omitted.\nGiven an instance ((U,F), k) of SET COVER, where U is a ground set of elements, F is a family of\nsubsets for U, and k \u2208 N is the parameter, we create an instance of CASCADING PORTFOLIO SCHE-\nDULING as follows. We set T = U, and for each F \u2208 F, we create an algorithm \u03b1F \u2208 A and add\n(\u03b1F , t) to S, for every t \u2208 F . Finally, we set (cid:96) = k. The function cost can be de\ufb01ned arbitrarily. The\nabove reduction is clearly a (polynomial-time) FPT-reduction, and it is straightforward to verify that\n((U,F), k) is a yes-instance of SET COVER if and only if the constructed instance of CASCADING\nPORTFOLIO SCHEDULING has a valid portfolio of size at most (cid:96).\nWe remark that the above construction can also be used to show that the problem variants arising in\nScenarios 1-3 described in the introduction remain W[2]-complete.\n4 Results for Degree Parameters\nThis section presents a classi\ufb01cation of the complexity of CASCADING PORTFOLIO SCHEDULING\nparameterized by the considered (success and failure) degree parameters.\nProposition 4. CPS[algorithm failure degree] is in FPT.\nf the algorithm failure degree, and let I = (A, T, cost, S) be an instance of\nProof. Denote by degA\nCASCADING PORTFOLIO SCHEDULING. Consider an algorithm which loops over each algorithm\n\u03b1 \u2208 A and proceeds under the assumption that \u03b1 is the \ufb01rst algorithm in an optimal valid portfolio.\nFor each such \u03b1, the number of tests in T that cannot be evaluated by \u03b1 is at most degA\nf . Removing\n\u03b1 from A and the subset of tests {t | (\u03b1, t) \u2208 S} from T results in an instance I\u2212 of CASCADING\nPORTFOLIO SCHEDULING with at most degA\nf tests, which, by Proposition 2, can be solved in time\nO\u2217((degA\nf ) to obtain an optimal solution for I\u2212. Pre\ufb01xing \u03b1 to the optimal solution obtained\nfor I\u2212 (assuming a solution exists) results in an optimal solution S\u03b1 for I under the constraint\nthat algorithm \u03b1 is the \ufb01rst algorithm. Enumeration every algorithm \u03b1 \u2208 A as the \ufb01rst algorithm,\ncomputing S\u03b1, and keeping track of the solution of minimum cost over all enumerations, results in\nan optimal solution for I. The running time of the above algorithm is O\u2217((degA\nProposition 5. CPS[test failure degree] is in FPT.\nf the test failure degree, and let I = (A, T, cost, S) be an instance of CASCA-\nProof. Denote by degT\nDING PORTFOLIO SCHEDULING. Consider an algorithm which (1) loops over each algorithm \u03b1 \u2208 A\nand proceeds under the assumption that \u03b1 is the last algorithm in an optimal valid portfolio \u03c4, and\nthen (2) loops over every test t in our instance and proceed under the assumption that t is a test that\nis solved only by \u03b1 in \u03c4. For each such choice of t and \u03b1, it follows that the algorithms preceding\n\u03b1 in \u03c4 do not solve t, and hence there are at most degT\nf many such algorithms. Therefore, we can\ncheck the validity and compute the cost of every possible ordered selection of a subset from these\nalgorithms that precede \u03b1 in \u03c4. After we \ufb01nish looping over all choices of \u03b1 and t, we output a valid\nportfolio of minimum cost.\n\nf )degA\nf ).\n\nf )degA\n\n5\n\n\ff )!).\n\nThere are |A| choices for a last algorithm \u03b1 and |T| choices for a desired test t. For each \ufb01xed \u03b1\nand t, there are at most O\u2217((degT\nf )!) many ordered selections of a subset of algorithms preceding \u03b1\nin \u03c4. It follows that the problem can be solved in time O\u2217((degT\nProposition 6. CPS[algorithm success degree], CPS[test success degree], and even CPS[algorithm\nsuccess degree + test success degree] are NP-hard already if the algorithm success degree is at most 3\nand test success degree is at most 2.\nProof. We reduce from the problem 3-MIN SUM VERTEX COVER, where we are given a graph\nH = (V, E) with maximum degree 3, and the task is to \ufb01nd a bijection \u03c3 : V \u2192 {1, . . . .V } that\ne\u2208E f\u03c3(e), where f\u03c3(e) = minv\u2208e \u03c3(v). Feige et al. [5] showed that there exists\n\u0001 > 0 such that it is NP-hard to approximate 3-MIN SUM VERTEX COVER within a ratio better\nthan 1 + \u0001. Given an instance of this problem, we construct an instance of (A, T, cost, S) of\nCASCADING PORTFOLIO SCHEDULING by letting A = V , adding for each edge e \u2208 E a test te\nto T , setting S = { (\u03b1, te) \u2208 A \u00d7 T : \u03b1 \u2208 e}, and setting cost(\u03b1, t) = 1 for all \u03b1 \u2208 A and t \u2208 T . It\ne\u2208E f\u03c3(e) are exactly those that give an ordering \u03c4\nof A of minimal cost. It remains to observe that the the algorithm success degree is 3 and the test\nsuccess degree is 2.\n5 Results for Cover Numbers\nIn this section we show that CPS[failure cover number] and CPS[success cover number] are both\n\ufb01xed-parameter tractable.\n\nminimizes(cid:80)\nis easy to verify that bijections \u03c3 that minimize(cid:80)\n\n5.1 Using the Failure Cover Number\n\nThe \ufb01rst of the two results follows from an even more general result, the \ufb01xed-parameter tractability\nof CPS[failure treewidth], where as the parameter we take the treewidth of the failure graph GI\nde\ufb01ned as follows.\nThe failure graph GI is a bipartite graph whose vertices consist of A \u222a T and where there is an edge\nbetween \u03b1 \u2208 A and t \u2208 T iff t fails on A. We note that the algorithm (or test) failure degree naturally\ncorresponds to the maximum degree in the respective bipartition of GI, and that the failure covering\nnumber is actually the size of a minimum vertex cover in GI.\nTreewidth [21, 8, 1] is a well-established graph parameter that measures the \u201ctree-likeness\u201d of\ninstances. Aside from treewidth, we will also need the notion of balanced separators in graphs. We\nintroduce these technical notions below.\nTreewidth and Separators. Let G = (V, E) be a graph. A tree decomposition of G is a pair\nXi\u2208V = V , and T is a rooted tree whose\nnode set is V, such that:\n1. For every edge {u, v} \u2208 E, there is an Xi \u2208 V, such that {u, v} \u2286 Xi; and\n2. for all Xi, Xj, Xk \u2208 V, if the node Xj lies on the path between the nodes Xi and Xk in the tree\n\n(V,T ) where V is a collection of subsets of V such that(cid:83)\n\nT , then Xi \u2229 Xk \u2286 Xj.\n\nThe width of the tree decomposition (V,T ) is de\ufb01ned to be max{|Xi| | Xi \u2208 V} \u2212 1. The treewidth\nof the graph G, denoted tw(G), is the minimum width over all tree decompositions of G.\nA pair of vertex subsets (A, B) is a separation in graph G if A \u222a B = V (G) and there is no edge\nbetween A \\ B and B \\ A. The separator of this separation is A \u2229 B, and the order of separation\n(A, B) is equal to |A \u2229 B|. We say that a separation (A, B) of G is an \u03b1-balanced separation if\n|A \\ B| \u2264 \u03b1|V (G)| and |B \\ A| \u2264 \u03b1|V (G)|.\nProof Strategy. Our main aim in this section will be to prove the following theorem:\nTheorem 7. CPS[failure treewidth] is in FPT.\n\nIt is easy to see that failure treewidth is at most the failure cover number plus 1 (consider, e.g., a tree\ndecomposition of the failure graph consisting of a sequence of bags, each containing the algorithms\nand tests forming the cover and one additional test or algorithm). Hence, once we establish Theorem 7\nwe obtain the following as an immediate corollary:\nCorollary 8. CPS[failure cover number] is in FPT.\n\n6\n\n\fWe \ufb01rst provide below a high-level overview of the proof of Theorem 7.\nWe solve the problem using dynamic programming on a tree decomposition of GI, by utilizing\nthe upper bound on the solution length derived in the \ufb01rst step. The running time is O\u2217(4tw(GI ) \u00b7\ntw(GI)tw(GI )). To make the dynamic programming approach work, for a current bag in the tree\ndecomposition, and for each test in the bag, we remember whether the test is solved by an algorithm\nin the future or by an algorithm in the past. Moreover, we remember which tests are solved by the\nsame algorithm. We also remember speci\ufb01cally which algorithm is the \u201c\ufb01rst\u201d from the future and\nwhich is the \u201c\ufb01rst\u201d from the past. Finally, we remember the relative positions of the algorithms in\nthe bag, the \ufb01rst algorithm from the future, the \ufb01rst algorithm from the past, and the algorithms that\nsolve the tests in the bag. Note that we do not remember which algorithms solve tests in the bag, only\ntheir relative position and whether they are in the past or future.\nWe now turn to giving a more detailed proof for Theorem 7.\n\nLemma 9. A minimum cost schedule for CASCADING PORTFOLIO SCHEDULING can be computed\nin time O\u2217(4tw \u00b7 twtw).\n\nProof Sketch. As with virtually all \ufb01xed-parameter algorithms parameterized by treewidth, we use\nleaf-to-root dynamic programming along a tree decomposition (in this case of the failure graph\nGI)\u2014see for instance the numerous examples presented in the literature [4, 3]. However, due to\nthe speci\ufb01c nature of our problem, the records dynamically computed by the program are far from\nstandard. This can already be seen by considering the size of our records: while most such dynamic\nprogramming algorithms only store records that have size bounded by a function of the treewidth, in\nour case the records will also have a polynomial dependence on m.\nAs a starting point, we will use the known algorithm of Bodlaender et al. [2] to compute a tree-\ndecomposition of width at most 5 \u00b7 tw(GI). We proceed by formalizing the used records. Let Xi be\na bag in the tree decomposition. A con\ufb01guration w.r.t. Xi is a tuple (\u03b1past, \u03b1future, \u03c3, \u03b4), where\n\u2022 \u03b1past is an algorithm that has been forgotten in a descendant of Xi,\n\u2022 \u03b1future is an algorithm that has not been introduced yet in Xi,\n\u2022 \u03c3 : Xi \u222a {\u03b1past, \u03b1future} \u2192 [|Xi| + 2], and\n\u2022 \u03b4 : T \u2229 Xi \u2192 {\u201cpast\u201d, \u201cfuture\u201d}.\nNote that there are at most 2|Xi| \u00b7 (|Xi| + 2)|Xi|+2 \u00b7 m2 = O\u2217(2tw \u00b7 twtw) con\ufb01gurations. The\ninterpretation of the con\ufb01guration is that \u03c3 tells us the relative positions in the \ufb01nal schedule of the\nalgorithms in Xi, \u03b1past, \u03b1future, and for each test in Xi the algorithm that \ufb01nally solves the test t. The\nfunction \u03b4, for a test t, tells us whether the algorithm that is the \ufb01rst in the schedule that solves t was\nalready introduced (\u201cpast\u201d) or will be introduced (\u201cfuture\u201d). The entry \u03b1past represents the speci\ufb01c\nalgorithm that is \ufb01rst in the schedule among all algorithms that have been already forgotten in the\ndescendant, and \u03b1future that among the ones that have not been introduced yet.\nWe say that a con\ufb01guration C = (\u03b1past, \u03b1future, \u03c3, \u03b4) w.r.t. Xi is admissible if\n\u2022 for all algorithms \u03b11, \u03b12 \u2208 A \u2229 (Xi \u222a {\u03b1past, \u03b1future}), it holds that \u03c3(\u03b11) (cid:54)= \u03c3(\u03b12);\n\u2022 for all t \u2208 T \u2229 Xi if \u03c3(t) = j, then for every j(cid:48) < j: if there is \u03b1 \u2208 A \u2229 (Xi \u222a {\u03b1past, \u03b1future})\n\u2022 for all t \u2208 T \u2229 Xi if \u03b4(t) = \u201cpast\u201d, then either \u03c3(\u03b1past) \u2264 \u03c3(t) or there is \u03b1 \u2208 A \u2229 Xi such that\n\u2022 for all t \u2208 T \u2229 Xi if \u03b4(t) = \u201cfuture\u201d, then \u03c3(\u03b1future) \u2264 \u03c3(t);\n\u2022 for all j(cid:48), j \u2208 [|Xi| + 2] such that j(cid:48) < j, if \u03c3\u22121(j(cid:48)) = \u2205, then \u03c3\u22121(j) = \u2205; and\n\u2022 if \u03c3(\u03b1) = \u03c3(t) for some \u03b1 \u2208 A \u2229 (Xi \u222a {\u03b1past}) and t \u2208 T \u2229 Xi, then \u03b4(t) = \u201cpast\u201d and \u03b1 solve\n\nsuch that \u03c3(\u03b1) = j(cid:48) then \u03b1 does not solve t;\n\n\u03c3(\u03b1) = \u03c3(t);\n\nt.\n\nNote that if we take any valid schedule, we can project it w.r.t. a bag Xi and obtain a con\ufb01guration\n(\u03b1past, \u03b1future, \u03c3, \u03b4). Such a con\ufb01guration will always be admissible and so we can restrict our attention\nto admissible con\ufb01gurations only. To simplify the notation we let \u0393i[C] = \u221e if C is not an admissible\ncon\ufb01guration w.r.t. Xi.\nNow for each Xi, we will compute a table \u0393i that contains an entry for each admissible con\ufb01guration\nC such that \u0393i[C] \u2208 N is the best cost, w.r.t. con\ufb01guration C, of the already introduced tests restricted\nto the already introduced algorithms and the algorithm \u03b1future.\n\n7\n\n\fClearly, the minimum cost schedule of the instance gives rise to some admissible con\ufb01guration C\nw.r.t. the root node Xr of the tree decomposition. Hence \u0393r[C] contains the minimum cost of a\nschedule. To complete the proof, it suf\ufb01ces to show how to update the records when traversing the\ntree-decomposition in dynamic fashion. Below, we list the sequence of claims (along with some\nexemplary proofs) used to this end.\nClaim 10. If Xi is a leaf node, then \u0393i can be computed in O(|\u0393i|) time.\nProof of Claim. Note that Xi = \u2205 and that none of the algorithms has been introduced in any leaf\nnode. The only admissible con\ufb01guration looks like (\u2205, \u03b1,{(\u03b1, 0)},\u2205), where \u03b1 \u2208 A. Moreover,\nsince no tests or algorithms were introduced at that point, the cost of all of these con\ufb01gurations is\nzero.\n\nk be a con\ufb01guration (\u03b1past, \u03b1future, \u03c31\n\nk, \u03b4p) such that \u03c31\n\nk(t) = k and let C 2\n\nk be a con\ufb01guration (\u03b1past, \u03b1future, \u03c32\n\nk(t) = k. Note that \u03c32\n\nk(x) = \u03c3(x) for all x \u2208 (Xi\u222a{\u03b1past, \u03b1future})\\{t} such that \u03c3(x) < k, \u03c32\n\nClaim 11. If Xi is an introduce node for a test with the only child Xj, then \u0393i can be computed in\nO(|\u0393i|) time.\nClaim 12. If Xi is an introduce node for an algorithm with the only child Xj, then \u0393i can be\ncomputed in O(|\u0393i|) time.\nClaim 13. If Xi is a forget node, which forgets a test t, with the only child Xj, then \u0393i can be\ncomputed in O((cid:96)|\u0393i|) time.\nProof of Claim. Let C = (\u03b1past, \u03b1future, \u03c3, \u03b4) be an admissible con\ufb01guration w.r.t. Xi. Forgetting\na test does not change the costs of introduced tests w.r.t. introduced algorithms. Hence, we only\nneed to \ufb01nd a con\ufb01guration w.r.t. Xj of the lowest cost that after removing t from \u03b4 results in\nC. Let \u03b4p be a function we get from \u03b4 by adding \u03b4p(t) = \u201cpast\u201d and let \u03b4f be a function we\nget from \u03b4 by adding \u03b4f (t) = \u201cfuture\u201d. First let Cf be a con\ufb01guration (\u03b1past, \u03b1future, \u03c3f , \u03b4f ) such\nthat \u03c3f (x) = \u03c3(x) for all x \u2208 (Xi \u222a {\u03b1past, \u03b1future}) \\ {t} and \u03c3f (t) = \u03c3(\u03b1future). Now, for\nk \u2208 [|Xi| + 2] and let C 1\nk(x) = \u03c3(x) for all\nx \u2208 (Xi \u222a {\u03b1past, \u03b1future}) \\ {t} and \u03c31\nk, \u03b4p)\nsuch that \u03c32\nk(x) = \u03c3(x)+1\nfor all x \u2208 (Xi \u222a {\u03b1past, \u03b1future}) \\ {t} such that \u03c3(x) \u2265 k, and \u03c31\nk would be\nalso shifted to \u03c3 after removing the entry for t.\nWe let \u0393i[C] be minimum among Cf and mink\u2208[|Xi|+2],(cid:96)\u2208{1,2} \u0393j[C (cid:96)\nk].\nClaim 14. If Xi is a forget node, which forgets an algorithm \u03b1, with the only child Xj, then \u0393i can\nbe computed in O(((cid:96) + m)|\u0393i|) time.\nProof of Claim. Let C = (\u03b1past, \u03b1future, \u03c3, \u03b4) be an admissible con\ufb01guration w.r.t. Xi. Clearly, when\nwe forget an algorithm, the cost of schedule given by \u03c3 w.r.t. already introduced algorithms and tests\ndoes not change. Hence, we just need to choose the best con\ufb01guration of Xj that can result in C.\nWe distinguish two cases depending on whether \u03b1past = \u03b1 or not.\nFirst, if \u03b1past = \u03b1, then for an already forgotten algorithm \u03b1(cid:48), k \u2208 [|Xi| + 2] such that \u03c3(\u03b1past) \u2265 k,\nand (cid:96) \u2208 {0, 1} let us denote by C\u03b1(cid:48),k,(cid:96) the con\ufb01guration (\u03b1(cid:48), \u03b1future, \u03c3(cid:96)\n\u03b1(cid:48),k(\u03b1(cid:48)) = k,\n\u03b1(cid:48),k, \u03b4) such that \u03c3(cid:96)\nfor all x \u2208 Xi \u222a {\u03b1past, \u03b1future} \u03c3(cid:96)\n\u03b1(cid:48),k(x) = \u03c3(x) if \u03c3(x) < k and \u03c3(cid:96)\n\u03b1(cid:48),k(x) = \u03c3(x) + (cid:96) otherwise.\n\u03b1(cid:48),k to be admissible, \u03c3\u22121(k) contains at least one test and no algorithm. In\nNote that in order for \u03c30\nthis case we let \u0393i[C] = min\u03b1(cid:48),k,(cid:96) \u0393j[C\u03b1(cid:48),k,(cid:96)].\nIf \u03b1past (cid:54)= \u03b1, then for k \u2208 [|Xi| + 2] such that \u03c3(\u03b1past) < k, and (cid:96) \u2208 {0, 1} let us denote by Ck,(cid:96) the\ncon\ufb01guration (\u03b1past, \u03b1future, \u03c3(cid:96)\nk(x) = \u03c3(x)\nk to be admissible,\nif \u03c3(x) < k and \u03c3(cid:96)\n\u03c3\u22121(k) contains at least one test and no algorithm. In this case we let \u0393i[C] = mink,(cid:96) \u0393j[Ck,(cid:96)].\nClaim 15. If Xi is a join node with children Xj1 and Xj2, then \u0393i can be computed from \u0393j1 and\n\u0393j2 in O(2(cid:96)m|\u0393i|) time.\nTo conclude, the last four claims show that it is possible to dynamically compute our records from the\nleaves of a nice tree decomposition to its root; once the records are known for the root, the algorithm\nhas all the information it needs to output with the solution.\n\nk(x) = \u03c3(x) + (cid:96) otherwise. Note that again in order for \u03c30\n\nk(\u03b1) = k, for all x \u2208 Xi\u222a{\u03b1past, \u03b1future} \u03c3(cid:96)\n\nk, \u03b4) such that \u03c3(cid:96)\n\nIt follows that CPS[failure treewidth] is \ufb01xed-parameter tractable, hence establishing Theorem 7.\n\n8\n\n\f5.2 Using the Success Cover Number\n\nThe aim of this section is to establish the \ufb01xed-parameter tractability of CPS[success cover number],\nwhich can be viewed as a dual result to Corollary 8. The techniques used to obtain this result are\nentirely different from those used in the previous subsection; in particular, the proof is based on a\nsigni\ufb01cant extension of the ideas introduced in the proof of Proposition 1.\nTheorem 16. CPS[success cover number] is in FPT.\nProof Sketch. Let I be an instance of CPS[covs]. Our \ufb01rst step is to compute a witness for the\nsuccess cover number covs, i.e., a set of algorithms A(cid:48) and tests T (cid:48) such that |A(cid:48) \u222a T (cid:48)| = covs and\neach pair in S has a non-empty intersection with A(cid:48) \u222a T (cid:48); as discussed in Subsection 2, this can be\ndone in polynomial time [7, Proposition 1]. Let V = 2A(cid:48)\u222aT (cid:48)\nbe the set of all subsets of covs. We will\nconstruct a directed arc-weighted graph D with vertex set V \u222a {x}, and with the property that each\nshortest path from \u2205 to x precisely corresponds to a minimum-cost schedule for the input instance I.\nIntuitively, reaching a vertex v in D which corresponds to a certain set of algorithms A0 and tests T0\nmeans that the schedule currently contains the algorithms in A0 plus an optimal choice of algorithms\nwhich can process the remaining tests in T0; information about the ordering inside the schedule is not\nencoded by the vertex v itself, but rather by the path from \u2205 to v.\nIn order to implement this idea, we will add the following arcs to D. To simplify the description, let\nA\u2217 be an arbitrary subset of A(cid:48) and T \u2217 be an arbitrary subset of T (cid:48). First of all, for each A\u2217 such that\nfor every test t \u2208 T \\ T (cid:48) there is some \u03b1 \u2208 A\u2217 satisfying (\u03b1, t) \u2208 S, we add the arc (A\u2217 \u222a T (cid:48), x)\nand assign it a weight of 0. This is done to indicate that A\u2217 \u222a T (cid:48) corresponds to a valid schedule.\nSecond, for each A\u2217 that is a proper subset of A(cid:48), \u03b10 \u2208 A(cid:48) \\ A\u2217, and T \u2217, we add the arc e from\nA\u2217 \u222a T \u2217 to A\u2217 \u222a {\u03b10} \u222a T \u2217 \u222a T0, where T0 contains every test t0 \u2208 T (cid:48) such that (\u03b10, t0) \u2208 S. In\norder to compute the weight of this arc e, we \ufb01rst compute the set Te of all tests outside of T \u2217 where\n\u03b10 will be queried (assuming \u03b10 is added to the schedule at this point); formally, t \u2208 Te if t (cid:54)\u2208 T \u2217\nand for each \u03b1(cid:48) \u2208 A\u2217 it holds that (\u03b1(cid:48), t) (cid:54)\u2208 S. For clarity, observe that T0 \u2286 Te. Now, we set the\n\nweight of e to(cid:80)\n\ncost(\u03b10, t).\n\nt(cid:54)\u2208T (cid:48):\u2200\u03b1\u2208A\u2217:(\u03b1,t)(cid:54)\u2208S cost(\u03b1\u03bb, t)(cid:1).\n\n(cid:0)(cid:80)\nt\u2208(T (cid:48)\\T \u2217) cost(\u03b1\u03bb, t)(cid:1) +(cid:0)(cid:80)\n\nt\u2208Te\nTo add our third and \ufb01nal set of edges, we \ufb01rst pre-compute for each T\u03bb \u2286 T (cid:48) \\ T \u2217 an algorithm\n\u03b1\u03bb \u2208 A \\ A(cid:48) such that:\n1. for each t\u03bb (cid:54)\u2208 T \u2217, (\u03b1\u03bb, t\u03bb) \u2208 S iff t\u03bb \u2208 T\u03bb (i.e., \u03b1\u03bb successfully solves exactly T\u03bb), and\n2. among all possible algorithms satisfying the above condition, \u03b1\u03bb achieves the minimum\ncost for all as-of-yet-unprocessed tests. Formally, \u03b1\u03bb minimizes the term price(\u03b1\u03bb) =\nNow, we add an arc e from each A\u2217 \u222a T \u2217 to each A\u2217 \u222a T \u2217 \u222a T\u03bb, where T\u03bb is de\ufb01ned as above and\nassociated with the test \u03b1\u03bb. The weight of e is precisely the value price(\u03b1\u03bb).\ns + 1 many vertices, a shortest path P from \u2205 to x in D can be\nNote that since the graph D has 2cov\ncomputed in time 2O(covs). Moreover, it is easy to verify that D can be constructed from an instance\nI in time at most 2O(covs) \u00b7 |I|2. At this point, it remains to verify that a shortest \u2205-x path P in D\ncan be used to obtain a solution for I.\n6 Conclusion\nWe studied the parameterized complexity of the CASCADING PORTFOLIO SCHEDULING problem\nunder various parameters. We identi\ufb01ed several settings where the NP-hardness of the problem can\nbe circumvented via exact \ufb01xed-parameter algorithms, including cases where (i) the algorithms have\na small failure degree, (ii) the tests have a small failure degree, (iii) the evaluation matrix has a small\nfailure cover, and (iv) the evaluation matrix has a small success cover. The \ufb01rst three cases can be\nseen as settings in which most algorithms succeed on most of the tests, whereas case (iv) can be seen\nas a setting where most algorithms fail.\nWe have complemented our algorithmic results with hardness results which allowed us to draw a\ndetailed complexity landscape of the problem. We would like to point out that all our hardness results\nhold even when all costs are unit costs. This \ufb01nding is signi\ufb01cant, as it reveals that the complexity of\nthe problem mainly depends on the success relation and not on the cost mapping.\nFor future work, it would be interesting to extend our study to the more complex setting where up to\np algorithms from the portfolio can be run in parallel. Here, the number p could be seen as a natural\nadditional parameter.\n\n9\n\n\fAcknowledgments\n\nRobert Ganian acknowledges the support by the Austrian Science Fund (FWF), Project P 31336, and\nis also af\ufb01liated with FI MUNI, Brno, Czech Republic. Stefan Szeider acknowledges the support by\nthe Austrian Science Fund (FWF), Project P 32441.\nReferences\n[1] H. L. Bodlaender. Discovering treewidth. In Proceedings of the 31st Conference on Current\nTrends in Theory and Practice of Computer Science (SOFSEM\u201905), volume 3381 of Lecture\nNotes in Computer Science, pages 1\u201316. Springer Verlag, 2005.\n\n[2] Hans L. Bodlaender, P\u00e5l Gr\u00f8n\u00e5s Drange, Markus S. Dregi, Fedor V. Fomin, Daniel Lokshtanov,\nand Michal Pilipczuk. A ckn 5-approximation algorithm for treewidth. SIAM J. Comput.,\n45(2):317\u2013378, 2016.\n\n[3] M. Cygan, F. Fomin, L. Kowalik, D. Lokshtanov, D. Marx, M. Pilipczuk, M. Pilipczuk, and\n\nS. Saurabh. Parameterized Algorithms. Springer, 2015.\n\n[4] Rodney G. Downey and Michael R. Fellows. Fundamentals of Parameterized Complexity. Texts\n\nin Computer Science. Springer, 2013.\n\n[5] Uriel Feige, L\u00e1szl\u00f3 Lov\u00e1sz, and Prasad Tetali. Approximating min sum set cover. Algorithmica,\n\n40(4):219\u2013234, 2004.\n\n[6] J\u00f6rg Flum and Martin Grohe. Parameterized Complexity Theory, volume XIV of Texts in\n\nTheoretical Computer Science. An EATCS Series. Springer Verlag, Berlin, 2006.\n\n[7] Robert Ganian, Iyad Kanj, Sebastian Ordyniak, and Stefan Szeider. Parameterized algorithms\nIn Proceeding of ICML, the Thirty-\ufb01fth International\nfor the matrix completion problem.\nConference on Machine Learning, Stockholm, July 10\u201315, 2018, pages 1642\u20131651. JMLR.org,\n2018. ISSN: 1938-7228.\n\n[8] Georg Gottlob, Reinhard Pichler, and Fang Wei. Bounded treewidth as a key to tractability of\n\nknowledge representation and reasoning. Arti\ufb01cial Intelligence, 174(1):105\u2013132, 2010.\n\n[9] Georg Gottlob and Stefan Szeider. Fixed-parameter algorithms for arti\ufb01cial intelligence,\nconstraint satisfaction, and database problems. The Computer Journal, 51(3):303\u2013325, 2006.\nSurvey paper.\n\n[10] Holger H. Hoos, Tom\u00e1\u0161 Peitl, Friedrich Slivovsky, and Stefan Szeider. Portfolio-based algorithm\nIn John N. Hooker, editor, Proceedings of CP 2018, the 24rd\nselection for circuit QBFs.\nInternational Conference on Principles and Practice of Constraint Programming, volume\n11008 of Lecture Notes in Computer Science, pages 195\u2013209. Springer Verlag, 2018.\n\n[11] Shinji Ito, Daisuke Hatano, Hanna Sumita, Akihiro Yabe, Takuro Fukunaga, Naonori Kakimura,\nand Ken-ichi Kawarabayashi. Regret bounds for online portfolio selection with a cardinality\nconstraint. In Advances in Neural Information Processing Systems 31: Annual Conference on\nNeural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montr\u00e9al,\nCanada., pages 10611\u201310620, 2018.\n\n[12] Pascal Kerschke, Holger H. Hoos, Frank Neumann, and Heike Trautmann. Automated algorithm\n\nselection: Survey and perspectives. Evolutionary Computation, pages 1\u201347, 2018.\n\n[13] Lars Kotthoff. Algorithm selection for combinatorial search problems: A survey. AI Magazine,\n\n35(3):48\u201360, 2014.\n\n[14] Marius Lindauer, Holger Hoos, Frank Hutter, and Kevin Leyton-Brown. Selection and con\ufb01gu-\nration of parallel portfolios. In Handbook of Parallel Constraint Reasoning., pages 583\u2013615.\n2018.\n\n[15] Marius Lindauer, Frank Hutter, Holger H. Hoos, and Torsten Schaub. Autofolio: An automati-\ncally con\ufb01gured algorithm selector (extended abstract). In Carles Sierra, editor, Proceedings\nof the Twenty-Sixth International Joint Conference on Arti\ufb01cial Intelligence, IJCAI 2017, Mel-\nbourne, Australia, August 19-25, 2017, pages 5025\u20135029. ijcai.org, 2017.\n\n10\n\n\f[16] Haipeng Luo, Chen-Yu Wei, and Kai Zheng. Ef\ufb01cient online portfolio with logarithmic regret.\nIn Advances in Neural Information Processing Systems 31: Annual Conference on Neural\nInformation Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montr\u00e9al, Canada.,\npages 8245\u20138255, 2018.\n\n[17] Rolf Niedermeier. Invitation to Fixed-Parameter Algorithms. Oxford Lecture Series in Mathe-\n\nmatics and its Applications. Oxford University Press, Oxford, 2006.\n\n[18] Luca Pulina and Armando Tacchella. A self-adaptive multi-engine solver for quanti\ufb01ed boolean\n\nformulas. Constraints, 14(1):80\u2013116, 2009.\n\n[19] John R. Rice. The algorithm selection problem. Advances in Computers, 15:65\u2013118, 1976.\n\n[20] Mattia Rizzini, Chris Fawcett, Mauro Vallati, Alfonso Emilio Gerevini, and Holger H. Hoos.\nStatic and dynamic portfolio methods for optimal planning: An empirical analysis. International\nJournal on Arti\ufb01cial Intelligence Tools, 26(1):1\u201327, 2017.\n\n[21] Neil Robertson and Paul D. Seymour. Graph minors. III. planar tree-width. J. Comb. Theory,\n\nSer. B, 36(1):49\u201364, 1984.\n\n[22] Olivier Roussel. Description of ppfolio 2012. In et al. A. Balint, editor, Proceedings of SAT\n\nChallenge 2012, page 47. University of Helsinki, 2012.\n\n[23] Matthew Streeter. Approximation algorithms for cascading prediction models. In Jennifer G.\nDy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine\nLearning, ICML 2018, Stockholmsm\u00e4ssan, Stockholm, Sweden, July 10-15, 2018, volume 80 of\nJMLR Workshop and Conference Proceedings, pages 4759\u20134767. JMLR.org, 2018.\n\n11\n\n\f", "award": [], "sourceid": 4172, "authors": [{"given_name": "Eduard", "family_name": "Eiben", "institution": "Royal Holloway, University of London"}, {"given_name": "Robert", "family_name": "Ganian", "institution": "TU Wien"}, {"given_name": "Iyad", "family_name": "Kanj", "institution": "DePaul University, Chicago"}, {"given_name": "Stefan", "family_name": "Szeider", "institution": "Vienna University of Technology"}]}