1. Performance prediction for real world optimisation problems Tommy Messelis Stefaan Haspeslagh Burak Bilgin Patrick De Causmaecker Greet Vanden Berghe

2. Domain Combinatorial optimisation trying to find the best / a good solution for a problem in a huge solution space space is too big for exhaustive search the use of metaheuristics is advised a heuristic is like a ‘rule of thumb’ a metaheuristic uses heuristics find good enough solutions in a reasonable amount of time

3. Nurse Rostering Problem Find an assignment of nurses to shifts given a set of constraints: hard constraints: e.g. nurse cannot work 2 shifts on the same day soft constraints: e.g. nurse should not work more than x days in a row the best solution is the assigment that respects all hard constraints and as many soft constraints as possible a constraint violation corresponds to a cost

4. Observations the same problem instance can be solved very fast by one algorithm, while another algorithm performs very bad for a given instance, one algorithm can find much better soltutions than another algorithm, while for another instance this can be opposite the same algorithm performs very differently on different instances →There is no single best algorithm to solve all problems of a given distribution.

5. Performance prediction When resources are scarce (e.g. just enough time to run one algorithm) it is of high importance to use them as efficiently as possible Good to know in advance how well an algorithm will do on a given instance without spending the resources based on properties of the problem instance

6. Empirical hardness models modelling the complexity of a problem instance distribution when solved with a particular solution method, measured by some performance criterion hardness is modelled as a function of features: readily available properties of the instances

7. Example Quality of an approximate solution obtained by a metaheuristic (measured as the accumulated costs associated with constraint violations) learn a mapping from the feature space onto this quality features: size properties constraint values: min & max consecutive working days aggregate funcions: tightness ratio (max / min) hardness ratio (availability / coverage demand) linear regression

8. Example

9. Results accurate models are built for different performance criteria based on basic properties of the problem instances: comparing predictions for different algorithms allows for selecting the best algorithm for a given instance (algorithm portfolio)