1. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 Analysis of a Benchmark Generator for the Reactive Scheduling Problem Amedeo Cesta 1, Nicola Policella 2, and Riccardo Rasconi 1 1 ISTC-cnr, Institute for Cognitive Science and Technology 2 ESA/ESOC, European Space Agency

2. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 2 Introduction Q8: Should the competition include benchmarks for dynamic scheduling problems, such as on-line scheduling and scheduling execution monitoring? –Reactive Scheduling Test-sets Generator (this talk) Our goal is to produce a General Framework for Project Scheduling Problems

3. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 3 Project Scheduling We focus our attention on Project Scheduling Scheduling is primarily concerned with figuring out WHEN tasks/activities should be executed so that the final solution guarantees “good performance” –Management of space missions –Transportation scheduling –Production chains in a factory Different techniques have been studied by many scientific communities, such as the Artificial Intelligence, Management Science and Operations Research

4. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 4 Project Scheduling Problems [RCPSP/max] t r1r1 t r2r2 resources c 2 =3 c 1 =2 resource constraints Project Activity Network temporal constraints [2, 5] 418 4 5 1 1 5 1 0 6 12 0 max separation

5. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 5 Schedule’s life is short! Unfortunately the synthesis of initially feasible solutions is hardly ever sufficient! –In real working environments, unforeseen events tend to quickly invalidate the schedules predictive assumptions Approaching a scheduling problem requires the coupling of –a predictive scheduling engine, able to propose a possible solution in a compact representation, and –a reactive scheduling engine, able to manage the current solution and to adjust the schedule at execution time

6. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 6 Broadening Project Scheduling Definition A broader definition of project scheduling problem, consists in the following two components: –the static sub-problem (or Predictive Scheduling): given a set of activities (or tasks) and a set of constraints (time and/or resource), it consists in computing a feasible assignment of start and end times for each activity. –the dynamic sub-problem (or Reactive Scheduling): it consists in monitoring the actual execution of the schedule and repairing the current solution (or producing brand new solutions), every time it is necessary. predictive scheduling solvers have been thoroughly evaluated through the production of several benchmark data sets and metrics the aspect related to reactive scheduling has not yet received the same level of attention HERE we define a benchmark generator!

7. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 7 Empirical Framework Testsets Generator Testsets Generator Project Scheduling Problem Predictive Scheduler Predictive Scheduler Reactive Scheduler Reactive Scheduler Initial Schedule Set of Exogenous Events Final Solution ------------------------------------- 2 {eventDelay a6 7 2} {eventDuration a2 5 4} -------------------------------------

8. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 8 Project Scheduling with Uncertainty temporal uncertainty resource uncertainty causal uncertainty Activities last longer than expected or they can be postponed A new precedence relation between a pair of activities requires a revision of previous choices Difference between nominal (left) and actual (right) resource availability. Reduction of resource availability blocks the execution of some activities an their consequent delay

9. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 9 Benchmark ingredients Activity delay aiai t aware ∆ st This element specifies the instant where the specific event is supposed to happen.

10. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 10 Benchmark ingredients Activity duration aiai t aware ∆ dur

11. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 11 Benchmark ingredients Change of resource availability t aware ∆ cap st ev et ev rjrj

12. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 12 Benchmark ingredients Change of activities set akak t aware akak r1r1 r2r2 est k let k dur k μ a = add req k = {1,2}

13. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 13 Benchmark ingredients Insertion/removal constraint a prec t aware a succ [d min, d max ] μ c = add

14. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 14 Benchmark ingredients 1.Activity delay, 2.Activity duration 3.Change of resource availability, 4.Change of activities set, 5.Insertion/removal constraint temporal causal resource

15. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 15 Instant Modifier To formally model an execution event we introduce the concept of Instant Modifier: –An Instant Modifier is an operator defined by a set of modifications Z and a time of execution t E, and whose application on the problem P produces a change of the problem at time t E. –Given a problem P the reactive scheduling problem is:

16. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 16 Testsets generator INPUT: –The scheduling problem –Number of events to generate –Probability of occurrence for each single type of event –The minimum and maximum magnitude of each type of event. OUTPUT: –Set of exogenous events SPACED in time Definition of consistent t aware, ------------------------------------- 2 {eventDelay a6 7 2} {eventDuration a2 5 4} -------------------------------------

17. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 17 Timing the Exogenous Events consistently t aware values determine the instants where each specific event is supposed to happen. –How to find consistent values for all possible executions? FIRST STEP: we add a set of simplifying assumptions on the events that have to be generated: –activities cannot be anticipated, –activity durations can only increase –there are only reduction of resource availability SECOND STEP: we used a relaxed version of the scheduling problem in which resource constraints are not taken into account. –This relaxed problem consists in a Simple Temporal Problem (STP) –This allows to compute the lower and the upper bound for the start and the end time of each activity

18. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 18 Timing the Exogenous Events consistently The assumptions guarantee the monotonic increase condition in the case of constrainedness. Limitation: it is not possible to model situations like activity anticipations or processing time reductions which entail constraints retractions.

19. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 19 Producing consistent t aware values in the case of a delay of activity a i (e delay ), t aware <= lb(st i ) in the case of change of duration of the activity a i (e dur ), t aware <= lb(et i ) in the case of adding/removing activity a k (e act ), t aware <= lb(st k ) if a k is removed t aware <= est k otherwise; in the case of adding/removing a constraint between a prec and a succ (e constr ), t aware <= lb(st prec ) if the constraint is removed t aware <= min(lb(st prec ), lb(st succ )) otherwise.

20. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 20 Further constraints the width of the delay on activity a i (e delay ), ∆ st <= ub(st i ) - lb(st i ) the change of activity duration (e dur ), ∆ dur <= ub(et i ) - lb(st i ) - p i the change of resource availability (e res ), 0 <= ∆ cap <= cap j

21. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 21 Tuning the Instances Difficulty It is fundamental to control the difficulty related to each generated event Use well known metrics to measure the structural properties of a problem before and after the insertion on an event  t e1e1 e2e2 e3e3 e4e4

22. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 22 Possible metrics Temporal Metrics Resource Metrics (Schwindt 1998) (Cesta et al. 1998) (Mastor 1970)

23. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 23 Example 8 activities 2 resources both with capacity 2 oddly numbered activities require one instance of r 1 while evenly numbered activities require one instance of r 2 all the oddly numbered activities have a start-time of at least 3 D= {4, 7, 4, 7, 3, 5, 3, 5} 2 resources both with capacity 2 ------------------------------------- 2 {eventDelay a6 7 2} {eventDuration a2 5 4} -------------------------------------

24. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 24 Conclusions The benchmark generator represents a fundamental means to foster: –Significant experimental analysis –Scheduling competition Benchmarks consist of a set of modification events –Type of modifications that can affect a schedule –To simulate the environmental uncertainty events are time spaced –It is worth to asses the difficulty of the instances Next step consists in the introduction of a “General Scheduling Execution framework” –Different combinations of proactive and reactive scheduling techniques can be evaluated

25. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 25 Conclusions and ongoing work initial step to design a complete experimental framework –introduce a set of metrics for (a)evaluating the validity of the different rescheduling techniques (b)having a measure to assess the difficulty of the testsets Study of scheduling approaches (both static and dynamic component) Research on reactive scheduling Testsets generation for Reactive Scheduling Metrics

26. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 26 Outline What’s reactive scheduling and why it is interesting? –Our goals Project Scheduling with Uncertainty –Ingredients for a benchmark sets A benchmark generator based on the RCPSP/max scheduling problem Long-term work –the design of a broad empirical framework for scheduling approaches

27. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 27 Project Scheduling with Uncertainty Different scheduling techniques have been developed to analyze scheduling problems and produce high quality solutions Those techniques usually consider the external world as static. Unfortunately, real world has a degree of uncertainty. In general, it is hard to have an exact estimation of its evolution A consequent brittleness of classical fixed time solutions

28. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 28 Reactive Scheduling A broader definition of scheduling problem, consisting in the following two components: –the static sub-problem (or Predictive Scheduling) given a set of activities (or tasks) and a set of constraints, it consists in computing a consistent assignment of start and end times for each activity. –the dynamic sub-problem (or Reactive Scheduling) it consists in monitoring the actual execution of the schedule and repairing the current solution (or producing brand new solutions), every time it is necessary. Even though the predictive scheduling aspects have been thoroughly evaluated through the production of several benchmark data sets and metrics, the aspect related to reactive scheduling has not yet received the same level of attention.

29. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 29 Testsets generator INPUT: –The scheduling problem –Number of events to generate –Probability of occurrence for each single type of event –The minimum and maximum magnitude of each type of event. Key point in a benchmark instance for the dynamic sub-problem is the fact that the different events have to be properly spaced in time This aspect has been taken into consideration with the definition of the parameter t aware, whose different values determine the instants where each specific event is supposed to happen. –How to define it?? –Finding a value which is consistent for all possible executions

30. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 30 Producing consistent t aware values However, we need to add a set of simplifying assumptions on the events that have to be generated: –activities cannot be anticipated, –activity durations can only increase –there are only reduction of resource availability These assumptions allow to rely on the lower bounds of the start times of each activity, lb(st i ), to define ``safe'' values for the t aware parameter related to the different events.

31. Analysis of a Benchmark Generator for the Reactive Scheduling Problem – Cesta, Policella, & Rasconi Scheduling a Scheduling Competition - ICAPS07 workshop, September 22, 2007 31 Long term work It is our opinion that research on the performance of reactive scheduling engines is fundamental –to assess reaction performances –to have an evaluation on the suitability and appropriateness of the scheduling approach in its entirety, that is, from the predictive and the reactive point of view. Study of scheduling approaches (both static and dynamic component) Research on reactive scheduling Testsets generation for Reactive Scheduling