1. Boolean Satisfiability The most fundamental NP-complete problem, and now a powerful technology for solving many real world problems

2. Overview CNF, SAT, 3SAT and 2SAT Resolution and Unit Propagation DPLL search Conflict-driven Nogood Learning Activity-based Search Modelling for SAT

3. Conjunctive Normal Form SAT solvers solve problems in –Conjunctive Normal Form (CNF) Boolean variable: b (true/false) or (0/1) Literal: l variable b or its negation –b –negating a literal: -l = -b if l = b, and -l = b if l = -b Clause: C disjunction or set of literals CNF: theory T set of clauses Assignment: set of literals A with {b,-b} not subset: e.g. {-b1,b2} Assign b1=false, b2=true

4. Boolean Satisfiability (SAT) This is the most basic NP-complete problem. SAT: Given a set of clauses T, find an assignment A such that for each C in T Each clause C in T is satisfied by A

5. 3SAT 3SAT: SAT with the restriction that each clause has length at most 3. SAT -> 3SAT –{l1, l2, l3, l4, l5, l6} l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6 becomes the set of clauses –{l1, l2, b1}, {-b1, l3, b2}, {-b2, l4, b3}, {-b3, l5, l6} where b1, b2, b3 are new Boolean variables 3SAT is NP-complete (by the above reduction)

6. 2SAT 2SAT: SAT with the restriction that each clause has length at most 2. 2SAT is decidable in polynomial time (n 3 ) 2SAT is NL-complete –which is a crazy complexity class –Nondeterministic Turing machine with log writeable memory!

7. Resolution The fundamental inference for CNF –{l1, l2, l3, b} {-b, l4, l5, l6} implies –{l1, l2, l3, l4, l5, l6} Or –(l1 ∨ l2 ∨ l3 ∨ b) ∧ ( -b ∨ l4 ∨ l5 ∨ l6) -> l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6 One can prove unsatisfiability of a CNF formula by repeatedly applying all possible resolutions –if this generates an empty clause then UNSAT –otherwise SAT Exponential process!

8. Unit Propagation Resolution = too expensive Unit Propagation (=unit resolution) –Restricted form of resolution = finds unit clauses {l1, l2, …, ln, l} where –li in A forall 1 ≤ i ≤ n Add l to assignment A {-b1, b2, b} A = {b1, -b2} –{-b1, b2, b} A := {b1, -b2, b} (-b1 ∨ b2 ∨ b) ∧ b1 ∧ -b2 -> b

9. Unit Propagation Repeatedly apply unit propagation, –until no new unit consequences can be found –or failure detected A = {b1, -b2, b3, -b4} C = {-b1, -b3, b4} {-b1, -b3, b4} Failure detected

10. Unit Propagation Example T = {{-e11,-e21}, {-e11,-e31},{-e11,-e41}, {e21,-b21}, {e31,-b31},{e41,-b41}, {b21,-b51},{b51,-b52,e52},{b41,-e52}, A = {e11,b52} {-e11,-e21} A ={e11,b52,-e21} {-e11,-e31} A ={e11,b52,-e21,-e31} {-e11,-e41} A ={e11,b52,-e21,-e31,-e41} {-e21,-b21} A ={e11,b52,-e21,-e31,-e41,-b21} {-e31,-b31} A ={e11,b52,-e21,-e31,-e41,-b21,-b31} {-e41,-b41} A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41} {b21,-b51} A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51} {b51,-b52,e52} A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51,e52} e11 -e21 -e31 -e41 -b21 -b31 -b41 -b51 b52 e52 fail {b41,-e52} A = false

11. Implication Graph Records the reason why each Boolean literal became true –Decision –Propagation(and why) Used for nogood reasoning! e11 -e21 -e31 -e41 -b21 -b31 -b41 -b51 b52 e52 fail

12. Unit Propagation Exercise T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5}, {b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,- b8,b10}, {-b5,-b10},{b5,b6,-b10},{-b8,b11}} A = {b1,-b3,b8} Perform unit propagation and draw the implication graph

13. Implication Graph e11 -e21 -e31 -e41 -b21 -b31 -b41 -b51 b52 -e52

14. DPLL search Davis-Putnam-Logemann-Loveland algorithm –interleave decisions + unit propagation dpll (A) A' = unitprop (A) if A' == false return false else if exists Boolean variable b not appearing in A if dpll (A' union {b}) return true else if dpll (A' union {-b}) return true else return false else return true

15. DPLL search with nogood learning dpll (A) A' = unitprop (A) if A' == false Add a clause C explaining the failure to T Backjump to the first place C can give new information else if exists Boolean variable b not appearing in A if dpll (A' union {b}) return true else if dpll (A' union {-b}) return true else return false else return true

16. Nogood Learning The implication graph shows a reason for failure Any cut separating the fail node from decisions –explains the failure e11 -e21 -e31 -e41 -b21 -b31 -b41 -b51 b52 e52 fail e52 ∧ -b41  false {b41,-e52} b52 ∧ -b51 ∧ -b41  false {b41,b51,-b52} b52 ∧ -b51 ∧ -e41  false {e41,b51,-b52} b52 ∧ e11  false {-e11,-b52}

17. Which Nogood? SAT solvers almost universally use the –First Unique Implication Point (1UIP) Nogood Closest nogood to failure –only one literal from the last decision level Asserting: on backjump it will unit propagate General: more general than many other choices Fast: doesn’t require too much computation –replace last literal in nogood until only one at last level

18. 1UIP Nogood Creation e11 -e21 -e31 -e41 -b21 -b31 -b41 -b51 b52 e52 b22e22 -e32 -e42 -b32 -b42 b33 b43 e33 e43 fail e33 ∧ e43  false {-e33,-e43} e43 e33 -b42 ∧ b43 ∧ e33  false {b42,-b43,-e33} b43 {-b32,-b42,b33,b43}  false {b32,b42,-b33,-b43} b33 {-b21,-b32,-b42,b33}  false {b21,b32,b42,-b33} -b32 -b21 -b42 {-b21,-b32,-b42}  false {b21,b32,b42} {-b21,-b41,-e42,-b32}  false {b21,b41,e42,b32} -e42-b41 {-b21,-b31,-b41,-e32,-e42}  false {b21,b31,b41,e32,e42} -e32-b31 {-b21,-b31,-b41,e22,-e32}  false {b21,b31,b41,-e22,e32} e22 {-b21,-b31,-b41,e22}  false {b21,b31,b41,-e22} 1 UIP Nogood

19. Backjumping Backtrack to second last level in nogood Nogood will propagate e.g. {b21,b31,b41,-e22} e11 -e21 -e31 -e41 -b21 -b31 -b41 -b51 -e22 {b21,b31,b41,-e22} Continue unit propagation then make next choice -b22 -b52

20. Why Add Nogoods We will not make the same choices again –{e11,b52} leads to failure –After choosing e11 we infer –b52 –Better yet, any choice that leads to b21,b31,b41 prevents the choice b52 Drastic reduction in search space Faster solving

21. 1UIP Exercise T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5}, {b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,- b8,b10}, {-b5,-b10},{b5,b6,-b10},{-b8,b11}} A = {b1,-b3,b8} Assume the order of the decisions was b8, -b3, b1 Determine the 1UIP nogood Where does execution backjump to?

22. Activity Each time a Boolean variable is seen during nogood creation, i.e. –appears in final nogood, or –is eliminated in the reverse propagation Increase activity by one These variables are helping cause failure Periodically divide all activities by some amount –activity reflects helping cause recent failure

23. Activity-based Search Select the unfixed variable with highest activity MiniSat: set to false RSAT: set to the last value it took –works with backjumping to recreate the same path –e.g. -b1, b3, -b11, b4, -b5, b7, fail –backjump to -b1, b3, -b11 –If b4 is now highest activity variable set it true [b4] –If b5 is next highest activity variable set it false [-b5] Activity-based search –concentrates on the variables causing failure –learns shorter nogoods by failing earlier

24. Activity-based Search Works well with restart On restart we concentrate on the now most active variables –a new part of the search space –learn new nogoods about this

25. Modern SAT Solvers Modern SAT solvers can handled problems with –(low) millions of clauses –millions of variables assuming the input has structure Random CNF is much harder (but uninteresting) Before the advent of nogood learning –(low) thousands of clauses –hundreds of variables

26. Modelling for SAT Boolean constructs can all be converted to clauses b1 = b2 ∨ b3 –{-b2,b1}, {-b3,b1}, {b2,b3,-b1} b1 = b2 ∧ b3 –{-b1,b2}, {-b1,b3}, {b1,-b2,-b3} b1 = (b2  b3) –{-b2,b3,b1}, {b2,-b3,b1}, {b2,b3,-b1}, {-b2,-b3,-b1} etc.

27. Modelling Integers in SAT Two main representations: i in 0..m (m = 2 k-1 – 1) Binary: i = 2 k-1 i k-1 + 2 k-2 i k-2 + … + 2i 1 + i 0 Unary: i = i m + i m-1 + … + i 1 –where i k  i k-1 Example representations IntegerBinaryUnary 1110110000011111111111 000000000000000000000 601100000000000111111

28. Pseudo-Boolean Constraints Psuedo-Boolean constraints: b i Boolean Cardinality constraints: Note that is b 1 ∨ … ∨ b m Straight clausal definitions are exponential Encode: Binary Decision Diagram, Binary or Unary arithmetic Examine cardinality constraint encodings

29. Cardinality Constraints (BDD) a  if b then c else d {-a,-b,c}, {-a,b,d}, {a,-b,-c}, {a,b,-d}  if then else

30. Cardinality Constraints (BDD)  if then else

31. Cardinality Constraints (BDD) true x 1 = 0x 1 = 1 x2x2 x3x3 Dashed arc: x i = 0 Full arc: x i = 1 Number of intermediate states is (n – k) ×k x4x4 x5x5 x1x1

32. BDD Exercise Draw the BDD for the cardinality constraint

33. Cardinality Constraints (binary) bibi  1   2   3   b i = (s k-1,…,s 0 ) s2s1s0s2s1s0 Sum(n)  Sum(n) 

34. Cardinality Constraints (binary) Sum(n) Sum(n/2)  1 Sum(n/2)  1  1   2   3   b i = (s k-1,…,s 0 ) Sum(n/2)  2 Sum(n/2)  2 s2s1s0s2s1s0 Binary adder  3 bibi

35. Binary Adder FA a 2 b 2 a 1 b 1 C2C2 C1C1 C in y 3 y 2 y 1 {y2,-a2,-b2,-c1}, {y2,-a2,b2,c1},{y2,a2,-b2,c1}, {y2,a2,b2,-c1}, {-y2,a2,b2,c1}, {-y2,a2,-b2,-c1}, {-y2,-a2,b2,-c1}, {-y2,-a2,-b2,c1}, {y3,-a2,-b2}, {y3,-a2,-c1}, {y3,-b2,-c1}, {-y3,a2,b2}, {-y3,a2,c1}, {-y3,b2,c1}

36. Cardinality Constraints (binary) bibi s2s1s0s2s1s0 Sum(n)  Sum(n)   b i ≤ 4 s 2   s 1 ∧  s 0 (s 2,s 1,s 0 ) ≤ 4 {-s 2,-s 1 }, {-s 2,-s 0 } How do you encode  b i ≤ 5

37. Sorting Networks A Comparator is a device with two inputs, a and b, and two outputs, c and d comparator a b c=max(a,b) d=min(a,b) A Sorting network is a composition of wires and comparators that sorts its inputs b1b1 a1a1 a2a2 b2b2 b3b3 a3a3 a4a4 b4b4 CNF of comparator {-a,c}, {-b,c}, {-a,-b,d}, {a,-d}, {b,-d}, {a,b,-c}

38. Sorting Networks There are “handcrafted” sorting networks And there are sorting networks based on your favorite sorting algorithm x1x1 x2x2 x3x3 x4x4 x8x8 y1y1 y2y2 y3y3 x5x5 x6x6 x7x7 y5y5 y6y6 y7y7 y8y8 x9x9 y9y9 Custom sorting network, n=9 x1x1 x2x2 x3x3 x4x4 y1y1 y2y2 y3y3 x5x5 x6x6 y5y5 y6y6 y4y4 y4y4 “Bubble sort” network 163245163245 613245613245 631245631245 632145632145 632415632415 654321654321

39. Odd-Even (OE) Sorting Networks x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 y1y1 y2y2 y3y3 y4y4 y5y5 y6y6 y7y7 y8y8 Sorter(n) Sorter(n/2) OE-Merger(n) Sorter(n/4 ) OE- Merger(n/2) Sorter(n/4 )

40. Odd-Even Merger OE-Merger(4) x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 y1y1 y2y2 y3y3 y5y5 y6y6 y7y7 y8y8 y4y4 Given two sorted input sequences sorted Merge the odd input sequences Merge the even input sequences OE Merger(8) OE-Merger(4)

41. Odd-Even Merger x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 y1y1 y2y2 y3y3 y5y5 y6y6 y7y7 y8y8 y4y4 Given two sorted input sequences sorted Merge the odd input sequences Merge the even input sequences Combine the outputs into a sorted output OE-Merger(4)

42. Sorting Networks Draw the best sorting network you can for 5 inputs! Minimize levels and/or comparators

43. Cardinality Constraints (unary) Sorting Network encodes unary sum Cardinality by setting output bits 01100110 11001100 10101010 10101010 10101010 x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 y1y1 y2y2 y3y3 y4y4 Sorter(8) 0 0 0 0

44. Cardinality Constraints BDD –quadratic size construction –Unit propagation on CNF enforces domain consistency Binary –Linear size construction –Unit Propagation on CNF does not enforce domain consistency Unary –n log 2 n size construction –unit propagation enforces domain consistency

45. Psuedo-Boolean Constraints Similar to Cardinality Constraints BDD –exponential in size, strong propagation Binary –O(n log n) in size, weak propagation Unary –O(n log 2 n) in size, medium propagation –interesting problem to find base for representation

46. Integer Constraints BDD –can handle arbitrary formula but exponential Unary –addition, subtraction, multiplication by constant –multiplication is impractical Binary –addition, subtraction –multiplication is quadratic

47. Boolean Solving in MiniZinc mzn –b sat invokes a SAT solver all constraints and variables must be Boolean Without recursion it’s a bit hard to model interesting problems with just MiniZinc

48. Example: Latin Squares int: n; % size array[1..n,1..n,1..n] of var bool: b; predicate atmostone(array[int] of var bool:x) = forall(i,j in index_set(x) where i < j)( (not x[i]) \/ (not x[j])); predicate exactlyone(array[int] of var bool:x) = atmostone(x) /\ exists(x); constraint forall(i,j in 1..n)( exactlyone([ b[i,j,k] | k in 1..n]) /\ exactlyone([ b[i,k,j] | k in 1..n]) /\ exactlyone([ b[k,i,j] | k in 1..n])); solve satisfy; output [ if fix(b[i,j,k]) then show(k) ++ if j == n then "\n" else " " endif else "" endif | i,j,k in 1..n]; 5 7 4 6 8 2 1 3 9 12 10 11 13 8 2 1 7 6 4 11 10 3 9 5 13 12 6 8 7 1 2 3 10 9 4 5 13 12 11 7 6 5 3 4 1 9 11 2 13 12 10 8 3 5 8 4 7 13 2 1 12 6 11 9 10 1 3 2 8 5 12 13 4 11 10 6 7 9 2 1 6 5 3 10 4 12 13 11 9 8 7 4 9 3 2 1 11 12 13 10 7 8 5 6 12 4 10 11 13 9 3 2 1 8 7 6 5 10 11 9 13 12 5 6 7 8 1 2 3 4 9 10 13 12 11 6 5 8 7 2 1 4 3 11 13 12 9 10 7 8 5 6 3 4 1 2 13 12 11 10 9 8 7 6 5 4 3 2 1 size 13

49. Summary Modern SAT solvers are powerful solvers –nogood learning –activity based search –restart State of the art solutions to Boolean problems Many CP problems which focus on alldifferent and ≠ can be mapped to Boolean problems –quasi-group completion (latin squares) –graph coloring

50. Exercise 1: atmostone There is a linear encoding of atmostone (x) –introduce a new array of sum variables s encoding s[i] = exists(j in 1..i)(x[i]) –s[1] = x[1] –if s[i] is true then x[i+1] must be false –s[i+1] can be defined in terms of s[i] and x[i+1] –its actually the sorting network circuit simplified Write a new predicate defn of atmostone (x) Compare the execution speed on latin squares