1. University of Toronto Mechanical & Industrial Engineering An Introduction to Constraint Programming Part II: Solving Scheduling Problems J. Christopher Beck Dept. of Mechanical & Industrial Engineering University of Toronto Canada

2. University of Toronto Mechanical & Industrial Engineering Where We Left Off … CP has been mostly developed to solve combinatorial feasibility and optimization problems General solution approach: tree search with inference at each node –no generic relaxation Key concept: global constraint propagation

3. University of Toronto Mechanical & Industrial Engineering Global Constraints A constraint over an arbitrary number of variables that embodies a frequently encountered sub- problem –all-different(v 1, …, v k ), global cardinality –disjunctive/cumulative resource –grammar –… [still some arguments about this within CP] Global Constraint Catalog www.emn.fr/z-info/sdemasse/gccat/index.html

4. University of Toronto Mechanical & Industrial Engineering Global Constraints A global constraint represents a commonly occurring problem substructure The focus of modeling and solving techniques –models are conjunctions of global constraints –solving is typically tree search with domain consistency enforced at every node

5. University of Toronto Mechanical & Industrial Engineering Differences with MIP Richer, extensible language Focus on global constraints Traditionally no explicit use of lower bounds –no optimization function constraint satisfaction problems –less focus on relaxations –less true now: LBs “inside” global constraints

6. University of Toronto Mechanical & Industrial Engineering When to Choose CP Finding a feasible solution is hard –intricate set of complicated constraints No tight relaxations Existing global constraints model parts of your problem Strong back-propagation

7. University of Toronto Mechanical & Industrial Engineering Questions?

8. University of Toronto Mechanical & Industrial Engineering Job Shop Scheduling Job Operation/Task/ Activity Precedence Constraint

9. University of Toronto Mechanical & Industrial Engineering Job Shop Scheduling makespan

10. University of Toronto Mechanical & Industrial Engineering Notation p j – processing time of activity j (aka duration) est j – earliest start time of activity j lst j – latest start time of activity j ect j – earliest completion time of activity j lct j – latest completion time of activity j pjpj est j lct j ect j lst j

11. University of Toronto Mechanical & Industrial Engineering Notation pjpj [est j lst j ][ect j lct j ] Domain of start times (represented by an interval)

12. University of Toronto Mechanical & Industrial Engineering Temporal Propagation (Arc Consistency - AC) 20151015 0 100 [0[20[35[4540]60]75]85] 100 20151015 0 [20[40[55[6540]60]75]85]20 The same as CPM

13. University of Toronto Mechanical & Industrial Engineering Simplified Edge-Finding (aka Constraint-Based Analysis (CBA)) 20 3580 15 9050 Operations on the same unary capacity resource What can you infer here?

14. University of Toronto Mechanical & Industrial Engineering Edge-Finding Exclusion 100 20 15 10 15 0 1080 S est(S) lct(S) 25

15. University of Toronto Mechanical & Industrial Engineering Edge-Finding Exclusion 100 20 15 10 15 0 10 75 Sest(S)lct(S) 25 70100

16. University of Toronto Mechanical & Industrial Engineering Exclusion Rules For all non-empty subsets, S, and activities A  S: (lft(S) - est(S) < dur A + dur(S))  (lft(S) - est A < dur A + dur(S)) est A  est(S) + dur(S) (lft(S) - est(S) < dur A + dur(S))  (lft A - est(S) < dur A + dur(S)) lft A  lft(S) - dur(S) On the same unary-capacity resource

17. University of Toronto Mechanical & Industrial Engineering Algorithm [Vilim et al 05] With a clever data structure (Θ-Λ-trees) a set of activities can be updated in O(n log n) time An easier-to-understand algorithm can do it in O(n 2 ) [Nuijten 1994]

18. University of Toronto Mechanical & Industrial Engineering CP vs. MIP for JSP Mean relative error vs. best known solution

19. University of Toronto Mechanical & Industrial Engineering Resource Allocation & Scheduling Assign jobs Schedule [Hooker 2005] Constraints, 10, 385-401, 2005.

20. University of Toronto Mechanical & Industrial Engineering CP Model [Heinz & Beck 2012] CPAIOR 2011. minimize cost of assignment all jobs assign to 1 resource global resource constraint bounds on start time var.

21. University of Toronto Mechanical & Industrial Engineering MIP Time-indexed Model resource capacity at each time point

22. University of Toronto Mechanical & Industrial Engineering Logic-Based Benders Decomp. Global Model Master Problem Subproblem 1Subproblem n... Solution Cut [Hooker & Ottosson 2003] Mathematical Programming, 96, 33-60, 2003.

23. University of Toronto Mechanical & Industrial Engineering The Approach Assign jobs Assign jobs to resources subject to sub-problem relaxation and Benders cuts Schedule independently Send Benders cuts back to master Repeat Assignment Cuts

24. University of Toronto Mechanical & Industrial Engineering LBBD Model MP SP SP relaxation Benders cuts global resource constraint [Hooker 2005] Constraints, 10, 385-401, 2005.

25. University of Toronto Mechanical & Industrial Engineering Results Number problems (out of 195) for which each method found: [Heinz & Beck 2012] CPAIOR 2012.

26. University of Toronto Mechanical & Industrial Engineering But wait, there’s more …

27. University of Toronto Mechanical & Industrial Engineering Better LBBD [Hooker 2007] Operations Research 55 2013. tighter SP relaxation

28. University of Toronto Mechanical & Industrial Engineering New MIP Model [Heinz, Ku, & Beck 2013] CPAIOR 2013. redundant variables use tighter SP relaxation from LBBS as a redundant resource capacity constraint

29. University of Toronto Mechanical & Industrial Engineering Results [Heinz, Ku, & Beck 2013] CPAIOR 2013.

30. University of Toronto Mechanical & Industrial Engineering Confused?

31. University of Toronto Mechanical & Industrial Engineering Conclusions from Experiments CP is worst? –well, it finds very high quality solutions is a very short time MIP is still a very strong technology even for scheduling problems Hybrids! –CP is a key part of LBBD –I didn’t show you the main point of the paper …

32. University of Toronto Mechanical & Industrial Engineering Full Results [Heinz, Ku, & Beck 2013] CPAIOR 2013. CIP = constraint integer programming

33. University of Toronto Mechanical & Industrial Engineering CP Hybrids

34. University of Toronto Mechanical & Industrial Engineering CP/OR Integration 1 OR inside –For years CP has been using OR algorithms “inside” global constraints edge-finding, all-diff, etc. OR Cost-based pruning in TSPTW

35. University of Toronto Mechanical & Industrial Engineering CP/OR Integration 2 Cooperating solvers, solving the same problem –Model the problem in CP, model the problem in MP, solvers communicate through the variables MP is “master” and CP supplies new implied constraints CP is “master” and MP supplies lower bounds, relaxed optimal solutions, etc. Higher level control: dynamic “master-slave” relationship ETSP

36. University of Toronto Mechanical & Industrial Engineering CP/OR Integration 3 Decomposition –Overall problem is divided between CP and MP solvers Benders decomposition (MP master, CP slave) Column generation Column generation Benders decomposition

37. University of Toronto Mechanical & Industrial Engineering CP/OR Integration 4 Deep integration –incorporation global constraint inference into a MIP search Constraint Integer Programming: SCIP SIMPL

38. University of Toronto Mechanical & Industrial Engineering MIE1619: Constraint Programming & Local Search J. Christopher Beck Dept. of Mechanical & Industrial Engineering University of Toronto Canada

39. University of Toronto Mechanical & Industrial Engineering Advanced Grad Course Course is for “doctoral stream” students This will be a challenging course –1-2 papers to read per week –10-15 hours/week outside lectures –I expect a high-level of performance both in class participation and in the project Expected level: publishable!

40. University of Toronto Mechanical & Industrial Engineering Goals Apply Constraint Programming (CP) and/or Local Search (LS) to your research interests Teach “grad school” skills –Literature research, algorithmic research, writing research papers, writing peer reviews, giving presentations, being an active member of the research community

41. University of Toronto Mechanical & Industrial Engineering Evaluation Project: Research Paper 60 5% checkpoint 5% initial submission 10% peer reviews 25% final paper 15% presentation Problem Presentation 15 1 of them 20 minutes + questions Modeling Assignments 15 number: enrollment - 1 10-questions10 Once or twice (in pairs)

42. University of Toronto Mechanical & Industrial Engineering Research Paper Goal Apply CP and/or LS to your research topic –I have some ideas for people who are stuck Publish! –The goal (but not a requirement) The paper you hand in for this course should be publishable And usable as part of your thesis

43. University of Toronto Mechanical & Industrial Engineering Lecture Goal Doing the readings is essential You are expected to participate each week –I will ask people to explain particular points –Starting in Week 3, you will begin preparing questions based on the readings

44. University of Toronto Mechanical & Industrial Engineering Outline Intro: 1 week CP: 3 weeks LS: 2 weeks Writing Peer Reviews: 0.5 weeks OR/CP/LS Hybrids: 4.5 weeks Project Presentations: 1 or 2 weeks

45. University of Toronto Mechanical & Industrial Engineering Questions?

46. University of Toronto Mechanical & Industrial Engineering Search: Texture Measurements Algorithms for the analysis of the constraint graph representation of a search state Heuristic search idea: –Use texture measurement to reveal problem structure –Formulate heuristic commitment based on the structure

47. University of Toronto Mechanical & Industrial Engineering The SumHeight Heuristic Find the resource, R*, and time point, t*, with highest competition Find the 2 activities not sequenced with each other with the highest individual demand for R* at t* Heuristically, post a sequencing constraint between them

48. University of Toronto Mechanical & Industrial Engineering The SumHeight Texture: Individual Demand time individual demand for R 2 estlctlstect A1A1

49. University of Toronto Mechanical & Industrial Engineering The SumHeight Texture: Aggregate Demand time aggregate demand for R 2 R2R2 R2R2 R2R2 Dynamic Focus of Attention

50. University of Toronto Mechanical & Industrial Engineering Search At each node (after propagation) –recalculate texture measurements –identify peaks –identify activity pair, A, B –branch on (A  B) OR (B  A) See [Beck 1999] for more than you want to know about this

51. University of Toronto Mechanical & Industrial Engineering What If We Want to Minimize Makespan? Assign jobs Schedule Assignments Cuts

52. University of Toronto Mechanical & Industrial Engineering Minimizing Makespan: What Changes? M = max j {t j +p x j j } different Benders cuts

53. University of Toronto Mechanical & Industrial Engineering The Challenge: Benders Cuts for Makespan Apparently this was the 12 th or 13 th Benders’ cut formulation that the author tried For details see [Hooker 2005]