1. Constraint Programming: modelling Toby Walsh NICTA and UNSW

2. Golomb rulers Mark ticks on a ruler  Distance between any two ticks (not just neighbouring ticks) is distinct Applications in radio-astronomy, cystallography, …  http://www.csplib.org/prob/prob006

3. Golomb rulers Simple solution  Exponentially long ruler  Ticks at 0,1,3,7,15,31,63,… Goal is to find minimal length rulers  turn optimization problem into sequence of satisfaction problems Is there a ruler of length m? Is there a ruler of length m-1? ….

4. Optimal Golomb rulers Known for up to 23 ticks Distributed internet project to find large rulers 0,1 0,1,3 0,1,4,6 0,1,4,9,11 0,1,4,10,12,17 0,1,4,10,18,23,25 Solutions grow as approximately O(n^2)

5. Modelling the Golomb ruler Variable, Xi for each tick Value is position on ruler Naïve model with quaternary constraints  For all i>j,k>l>j |Xi-Xj| \= |Xk-Xl|

6. Problems with naïve model Large number of quaternary constraints  O(n^4) constraints Looseness of quaternary constraints  Many values satisfy |Xi-Xj| \= |Xk-Xl|  Limited pruning

7. A better non-binary model Introduce auxiliary variables for inter-tick distances  Dij = |Xi-Xj|  O(n^2) ternary constraints Post single large non-binary constraint  alldifferent([D11,D12,…]).  Tighter constraints and denser constraint graph

8. Other modeling issues Symmetry  A ruler can always be reversed!  Break this symmetry by adding constraint: D12 < Dn-1,n  Also break symmetry on Xi X1 < X2 < … Xn  Such tricks important in many problems

9. Other modelling issues Additional (implied) constraints  Don’t change set of solutions  But may reduce search significantly E.g. D12 < D13, D23 < D24, … E.g. D1k at least sum of first k integers Pure declarative specifications are not enough!

10. Solving issues Labeling strategies often very important  Smallest domain often good idea  Focuses on “hardest” part of problem Best strategy for Golomb ruler is instantiate variables in strict order  Heuristics like fail-first (smallest domain) not effective on this problem!

11. Experimental results Runtime/secNaïve modelAlldifferent model 8-Find2.00.1 8-Prove12.010.2 9-Find31.71.6 9-Prove1689.7 10-Find65724.3 10-Prove> 10^568.3

12. Something to try at home? Circular (or modular) Golomb rulers  Inter-tick distance variables more central, removing rotational symmetry? 2-d Golomb rulers All examples of “graceful” graphs

13. Summary Modelling decisions:  Auxiliary variables  Implied constraints  Symmetry breaking constraints More to constraints than just declarative problem specifications!

14. Case study 2: all interval series

15. All interval series Prob007 at www.csplib.orgwww.csplib.org Comes from musical composition  Traced back to Alban Berg  Extensively used by Ernst Krenek Op.170 “Quaestio temporis”

16. All interval series Take the 12 standard pitch classes  c, c#, d,..  Represent them by numbers 0,.., 11 Find a sequence so each occurs once  Each difference occurs once

17. All interval series Can generalize to any n (not just 12) Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are all distinct Finding one solution is easy

18. All interval series Can generalize to any n (not just 12) Find Sn, a permutation of [0,n) such that |Sn+1-Sn| are all distinct Finding one solution is easy [n,1,n-1,2,n-2,.., floor(n/2)+2,floor(n/2)-1,floor(n/2)+1,floor(n/2)] Giving the differences [n-1,n-2,..,2,1] Challenge is to find all solutions!

19. Basic methodology Devise basic CSP model  What are the variables? What are the constraints? Introduce auxiliary variables if needed Consider dual or combined models Break symmetry Introduce implied constraints

20. Basic CSP model What are the variables?

21. Basic CSP model What are the variables? Si = j if the ith note is j What are the constraints?

22. Basic CSP model What are the variables? Si = j if the ith note is j What are the constraints? Si in [0,n) All-different([S1,S2,… Sn]) Forall i<i’ |Si+1 - Si| =/ |Si’+1 - Si’| Will this model be any good? If so, why? If not, why not?

23. Basic methodology Devise basic CSP model  What are the variables? What are the constraints? Introduce auxiliary variables if needed Consider dual or combined models Break symmetry Introduce implied constraints

24. Improving basic CSP model Is it worth introducing any auxiliary variables?  Are there any loose or messy constraints we could better (more compactly?) express via some auxiliary variables?

25. Improving basic CSP model Is it worth introducing any auxiliary variables?  Yes, variables for the pairwise differences Di = |Si+1 - Si| Now post single large all-different constraint Di in [1,n-1] All-different([D1,D2,…Dn-1])

26. Basic methodology Devise basic CSP model  What are the variables? What are the constraints? Introduce auxiliary variables if needed Consider dual or combined models Break symmetry Introduce implied constraints

27. Break symmetry Does the problem have any symmetry?

28. Break symmetry Does the problem have any symmetry?  Yes, we can reverse any sequence S1, S2, … Sn is an all-inverse series Sn, …, S2, S1 is also How do we eliminate this symmetry?

29. Break symmetry Does the problem have any symmetry?  Yes, we can reverse any sequence S1, S2, …, Sn is an all-inverse series Sn, …, S2, S1 is also How do we eliminate this symmetry? As with Golomb ruler! D1 < Dn-1

30. Break symmetry Does the problem have any other symmetry?

31. Break symmetry Does the problem have any other symmetry?  Yes, we can invert the numbers in any sequence 0, n-1, 1, n-2, … map x onto n-1-x n-1, 0, n-2, 1, … How do we eliminate this symmetry?

32. Break symmetry Does the problem have any other symmetry?  Yes, we can invert the numbers in any sequence 0, n-1, 1, n-2, … map x onto n-1-x n-1, 0, n-2, 1, … How do we eliminate this symmetry? S1 < S2

33. Basic methodology Devise basic CSP model  What are the variables? What are the constraints? Introduce auxiliary variables if needed Consider dual or combined models Break symmetry Introduce implied constraints

34. Implied constraints Are there useful implied constraints to add?

35. Implied constraints Are there useful implied constraints to add?  Hmm, unlike Golomb ruler, we only have neighbouring differences  So, no need to consider transitive closure

36. Implied constraints Are there useful implied constraints to add?  Hmm, unlike Golomb ruler, we are not optimizing  So, no need to improve propagation for optimization variable

37. Performance Basic model is poor Refined model able to compute all solutions up to n=14 or so  GAC on all-different constraints very beneficial  As is enforcing GAC on Di = |Si+1-Si| This becomes too expensive for large n So use just bounds consistency (BC) for larger n

38. Case study 3: orthogonal Latin squares

39. Modelling decisions Many different ways to model even simple problems Combining models can be effective  Channel between models Need additional constraints  Symmetry breaking  Implied (but logically) redundant

40. Latin square Each colour appears once on each row Each colour appears once on each column Used in experimental design  Six people  Six one-week drug trials

41. Orthogonal Latin squares Find a pair of Latin squares  Every cell has a different pair of elements Generalized form:  Find a set of m Latin squares  Each possible pair is orthogonal

42. Orthogonal Latin squares 1 2 3 4 2 1 4 3 3 4 1 2 3 4 1 2 4 3 2 1 4 3 2 1 2 1 4 3 11 22 33 44 23 14 41 32 34 43 12 21 42 31 24 13 Two 4 by 4 Latin squares No pair is repeated

43. History of (orthogonal) Latin squares Introduced by Euler in 1783  Also called Graeco-Latin or Euler squares No orthogonal Latin square of order 2  There are only 2 (non)-isomorphic Latin squares of order 2 and they are not orthogonal

44. History of (orthogonal) Latin squares Euler conjectured in 1783 that there are no orthogonal Latin squares of order 4n+2  Constructions exist for 4n and for 2n+1  Took till 1900 to show conjecture for n=1  Took till 1960 to show false for all n>1 6 by 6 problem also known as the 36 officer problem “… Can a delegation of six regiments, each of which sends a colonel, a lieutenant-colonel, a major, a captain, a lieutenant, and a sub- lieutenant be arranged in a regular 6 by 6 array such that no row or column duplicates a rank or a regiment?”

45. More background Lam’s problem  Existence of finite projective plane of order 10  Equivalent to set of 9 mutually orthogonal Latin squares of order 10  In 1989, this was shown not to be possible after 2000 hours on a Cray (and some major maths) Orthogonal Latin squares also used in experimental design

46. A simple 0/1 model Suitable for integer programming  Xijkl = 1 if pair (i,j) is in row k column l, 0 otherwise  Avoiding advice never to use more than 3 subscripts! Constraints  Each row contains one number in each square Sum_jl Xijkl = 1 Sum_il Xijkl = 1  Each col contains one number in each square Sum_jk Xijkl = 1 Sum_ik Xijkl = 1

47. A simple 0/1 model Additional constraints  Every pair of numbers occurs exactly once Sum_kl Xijkl = 1  Every cell contains exactly one pair of numbers Sum_ij Xijkl = 1 Is there any symmetry?

48. Symmetry removal Important for solving CSPs  Especially for proofs of optimality? Orthogonal Latin square has lots of symmetry  Permute the rows  Permute the cols  Permute the numbers 1 to n in each square How can we eliminate such symmetry?

49. Symmetry removal Fix first row 11 22 33 … Fix first column 11 23 32.. Eliminates all symmetry?

50. What about a CSP model? Exploit large finite domains possible in CSPs  Reduce number of variables  O(n^4) -> ? Exploit non-binary constraints  Problem states that squares contain pairs that are all different  All-different is a non-binary constraint our solvers can reason with efficiently

51. CSP model 2 sets of variables  Skl = i if the 1st element in row k col l is i  Tkl = j if the 2nd element in row k col l is j How do we specify all pairs are different?  All distinct (k,l), (k’,l’) if Skl = i and Tkl = j then Sk’l’=/ i or Tk’l’ =/ j O(n^4) loose constraints, little constraint propagation! What can we do?

52. CSP model Introduce auxiliary variables  Fewer constraints, O(n^2)  Tightens constraint graph => more propagation  Pkl = i*n + j if row k col l contains the pair i,j Constraints  2n all-different constraints on Skl, and on Tkl  All-different constraint on Pkl  Channelling constraint to link Pkl to Skl and Tkl

53. CSP model v O/1 model CSP model  3n^2 variables  Domains of size n, n and n^2+n  O(n^2) constraints  Large and tight non- binary constraints 0/1 model  n^4 variables  Domains of size 2  O(n^4) constraints  Loose but linear constraints Use IP solver!

54. Solving choices for CSP model Variables to assign  Skl and Tkl, or Pkl? Variable and value ordering How to treat all-different constraint  GAC using Regin’s algorithm O(n^4)  AC using the binary decomposition

55. Good choices for the CSP model Experience and small instances suggest:  Assign the Skl and Tkl variables  Choose variable to assign with Fail First (smallest domain) heuristic Break ties by alternating between Skl and Tkl  Use GAC on all-different constraints for Skl and Tkl  Use AC on binary decomposition of large all- different constraint on Pkl

56. Performance n0-1 model Fails t/sec CSP model AC Fails t/sec CSP model GAC Fails t/sec 44 0.112 0.182 0.38 51950 4.05295 1.39190 1.55 6? 640235 657442059 773 7*20083 59.891687 51.157495 66.1

57. Case study 4: Langford’s problem

58. Langford’s problem Prob024 @ www.csplib.org www.csplib.org Find a sequence of 8 numbers  Each number [1,4] occurs twice  Two occurrences of i are i numbers apart Unique solution  41312432

59. Langford’s problem L(k,n) problem  To find a sequence of k*n numbers [1,n]  Each of the k successive occrrences of i are i apart  We just saw L(2,4) Due to the mathematician Dudley Langford  Watched his son build a tower which solved L(2,3)

60. Langford’s problem L(2,3) and L(2,4) have unique solutions L(2,4n) and L(2,4n-1) have solutions  L(2,4n-2) and L(2,4n-3) do not  Computing all solutions of L(2,19) took 2.5 years! L(3,n)  No solutions: 0<n<8, 10<n<17, 20,..  Solutions: 9,10,17,18,19,.. A014552 Sequence: 0,0,1,1,0,0,26,150,0,0,17792,108144,0,0,39809640,326721800, 0,0,256814891280,2636337861200

61. Basic model What are the variables?

62. Basic model What are the variables? Variable for each occurrence of a number X11 is 1st occurrence of 1 X21 is 1st occurrence of 2.. X12 is 2nd occurrence of 1 X22 is 2nd occurrence of 2.. Value is position in the sequence

63. Basic model What are the constraints?  Xij in [1,n*k]  Xij+1 = i+Xij  Alldifferent([X11,..Xn1,X12,..Xn2,..,X1k,..Xnk ])

64. Recipe Create a basic model  Decide on the variables Introduce auxiliary variables  For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver  Variable, value ordering

65. Break symmetry Does the problem have any symmetry?

66. Break symmetry Does the problem have any symmetry?  Of course, we can invert any sequence!

67. Break symmetry How do we break this symmetry?

68. Break symmetry How do we break this symmetry?  Many possible ways  For example, for L(3,9) Either X92 < 14 (2nd occurrence of 9 is in 1st half) Or X92=14 and X82<14 (2nd occurrence of 8 is in 1st half)

69. Recipe Create a basic model  Decide on the variables Introduce auxiliary variables  For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver  Variable, value ordering

70. What about dual model? Can we take a dual view?

71. What about dual model? Can we take a dual view? Of course we can, it’s a permutation!

72. Dual model What are the variables?  Variable for each position i What are the values?

73. Dual model What are the variables?  Variable for each position i What are the values?  If use the number at that position, we cannot use an all-different constraint  Each number occurs not once but k times

74. Dual model What are the variables?  Variable for each position i What are the values?  Solution 1: use values from [1,n*k] with the value i*n+j standing for the ith occurrence of j  Now want to find a permutation of these numbers subject to the distance constraint

75. Dual model What are the variables?  Variable for each position i What are the values?  Solution 2: use as values the numbers [1,n]  Each number occurs exactly k times  Fortunately, there is a generalization of all-different called the global cardinality constraint (gcc) for this

76. Global cardinality constraint Gcc([X1,..Xn],l,u) enforces values used by Xi to occur between l and u times  All-different([X1,..Xn]) = Gcc([X1,..Xn],1,1) Regin’s algorithm enforces GAC on Gcc in O(n^2.d)  Regin’s papers are tough to follow but this seems to beat his algorithm for all-different!?

77. Dual model What are the constraints?  Gcc([D1,…Dk*n],k,k)  Distance constraints?

78. Dual model What are the constraints?  Gcc([D1,…Dk*n],k,k)  Distance constraints: Di=j then Di+j+1=j

79. Combined model Primal and dual variables Channelling to link them  What do the channelling constraints look like?

80. Combined model Primal and dual variables Channelling to link them  Xij=k implies Dk=i

81. Solving choices? Which variables to assign?  Xij or Di

82. Solving choices? Which variables to assign?  Xij or Di, doesn’t seem to matter Which variable ordering heuristic?  Fail First or Lex?

83. Solving choices? Which variables to assign?  Xij or Di, doesn’t seem to matter Which variable ordering heuristic?  Fail First very marginally better than Lex

84. Recipe Create a basic model  Decide on the variables Introduce auxiliary variables  For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver  Variable, value ordering