1. ©2007 Tarik Hadzic1 Lecture 11: Consistency Techniques 1. Arc Consistency 2. Directional Consistency 3. Generalized Arc Consistency Efficient AI Programming

2. ©2007 Tarik Hadzic2 Today’s Program Arc-Consistency - AC-1 - AC-3, - AC-4 Directional Consistency –DAC-1 –Backtrack-free search Global Constraints – Generalized Arc Consistency – Domain Store Propagation

3. ©2007 Tarik Hadzic3 Bounded Constraint Inference Guarantee: any consistent instantiation of i-1 variable is extendible to any i-th variable Algorithms: –i=2: Arc-Consistency –i=3: Path-Consistency –i≥4: i-consistency i-consistency algorithms exponential in i. Trade-off between preprocessing and subsequent search

4. ©2007 Tarik Hadzic4 Arc-Consistency

5. ©2007 Tarik Hadzic5 Arc-Consistency Guarantee: any value in the domain of a single variable can be extended consistently by any other variable Example: network X={x,y}, R xy : x<y, D x =D y ={1,2,3} 1 2 3 1 2 3 x < y x y

6. ©2007 Tarik Hadzic6 Example Network X={x,y}, R xy : x<y, D x =D y ={1,2,3} Not Arc-Consistent y 1 2 2 3 x < y x 1 2 3 1 2 3 x y Arc-Consistent

7. ©2007 Tarik Hadzic7 Definition Given constraint problem and constraint C ij (S ij,R ij ) in C An arc (x i, x j ) is arc-consistent iff for every a i  D i there exists a j  D j such that (a i,a j )  R ij If (x i, x j ) is arc-consistent, it doesn’t imply that (x j, x i ) is arc-consistent. A CSP is arc-consistent iff all of its arcs are arc-consistent

8. ©2007 Tarik Hadzic8 Revise ((x i ),x j ) 1.deleted  false 2. for each a i  D i 3. if there is no a j  D j, (a i,a j )  R ij 4. then delete a i from D, deleted  true 5. endif 6. endfor 7. return deleted Complexity? O(k 2 ) where k bounds the domain size

9. ©2007 Tarik Hadzic9 Revise ((x i ),x j ) 1 2 1 2 3 x < y x y 1 2 3 2 3 x y 1 2 3 1 2 3 x y Revise((x),y) Revise((y),x)

10. ©2007 Tarik Hadzic10 Revise ((x i ),x j ) To achieve arc-consistency, apply Revise until no change in the domain of any variable in the network x y z Revise((x),z)

11. ©2007 Tarik Hadzic11 Revise ((x i ),x j ) To achieve arc-consistency, apply Revise until no change in the domain of any variable in the network x y z Revise((x),z) Revise((y),x)

12. ©2007 Tarik Hadzic12 Revise ((x i ),x j ) To achieve arc-consistency, apply Revise until no change in the domain of any variable in the network x y z Revise((x),z) Revise((z),y) Revise((y),x)

13. ©2007 Tarik Hadzic13 Revise ((x i ),x j ) To achieve arc-consistency, apply Revise until no change in the domain of any variable in the network x y z Revise((x),z) Revise((z),y) Revise((x),z) Revise((y),x)

14. ©2007 Tarik Hadzic14 Revise ((x i ),x j ) To achieve arc-consistency, apply Revise until no change in the domain of any variable in the network x y z Revise((x),z) Revise((z),y) Revise((x),z) Revise((y),x)

15. ©2007 Tarik Hadzic15 Revise ((x i ),x j ) To achieve arc-consistency, apply Revise until no change in the domain of any variable in the network x y z Revise((x),z) Revise((z),y) Revise((x),z) Revise((y),x) Revise((z),y) Revise((y),x)

16. ©2007 Tarik Hadzic16 Revise ((x i ),x j ) To achieve arc-consistency, apply Revise until no change in the domain of any variable in the network x y z Revise((x),z) Revise((z),y) Revise((x),z) Revise((y),x) Revise((z),y) Revise((y),x)

17. ©2007 Tarik Hadzic17 AC-1 (  ) 1. repeat 2. for every {x i,x j } participating in a constraint 3. Revise((x i ),x j ) 4. Revise((x j ),x i ) 5. endfor 6. until no domain changed Complexity? O(enk 3 ) where n variables, e binary constraints, k bounds the domain size

18. ©2007 Tarik Hadzic18 AC-3 (  ) Improvement over AC-1 –No need to process all constraints –Revise((x i ),x j ) is processed only when D j is reduced –queue maintains pairs (x i,x j ) involved in a constraint where D j is reduced –Initially, for every constraint C ij, (x i,x j ) and (x j,x i ) are added to queue

19. ©2007 Tarik Hadzic19 AC-3 (  ) 1. for every constraint C ij 2. Q ← Q U {(x i,x j ),(x j,x i )} 3. endfor 4. while Q ≠ {} 5. select and delete (x i,x j ) from Q 6. Revise((x i ),x j ) 7. if Revise((x i ),x j )=true then 8. Q ← Q U {(x k,x i ), k ≠i, k≠j} 9. endif 10. endwhile

20. ©2007 Tarik Hadzic20 Example: AC-3 (  ) X = {x,y,z}, D z ={2,5}, D x ={2,5}, D y ={2,4}; R zx : ”z divides x”; R y z : “z divides y” 2,5 2,4 x z y Q = {(z,x),(x,z),(z,y),(y,z)}

21. ©2007 Tarik Hadzic21 Example: AC-3 (  ) X = {x,y,z}, D z ={2,5}, D x ={2,5}, D y ={2,4}; R zx : ”z divides x”; R xz : “z divides y” 2,5 2,4 x z y Q = {(z,x),(x,z),(z,y),(y,z)} Revise((z),x) No effect!

22. ©2007 Tarik Hadzic22 Example: AC-3 (  ) X = {x,y,z}, D z ={2,5}, D x ={2,5}, D y ={2,4}; R zx : ”z divides x”; R xz : “z divides y” 2,5 2,4 x z y Q = {(x,z),(z,y),(y,z)} Revise((x),z) No effect!

23. ©2007 Tarik Hadzic23 Example: AC-3 (  ) X = {x,y,z}, D z ={2,5}, D x ={2,5}, D y ={2,4}; R zx : ”z divides x”; R yz : “z divides y” 2 2,5 2,4 x z y Q = {(y,z),(x,z)} Revise((y),z) No effect!

24. ©2007 Tarik Hadzic24 Example: AC-3 (  ) X = {x,y,z}, D z ={2,5}, D x ={2,5}, D y ={2,4}; R zx : ”z divides x”; R xz : “z divides y” 2 2,5 2,4 x z y Q = {(x,z)} Revise((x),z) Delete 5 from Dx No constraints added to queue!

25. ©2007 Tarik Hadzic25 Example: AC-3 (  ) X = {x,y,z}, D z ={2,5}, D x ={2,5}, D y ={2,4}; R zx : ”z divides x”; R xz : “z divides y” 2 2 2,4 x z y Q = {} AC-3 terminates!

26. ©2007 Tarik Hadzic26 AC-4 (  ) Optimal performance O(ek 2 ) Does not use Revise For every value a  D i maintains counter(x i, x j, a) : number of values in Dj consistent with x i =a 2,5 2,4 x z y counter(z,y,2) = counter(z,y,5) = counter(z,x,2) = counter(z,x,5) = 2 0! 1 1 No counters between x and y

27. ©2007 Tarik Hadzic27 AC-4 (  ) If counter(x i,x j,a)=0 then (x i,a) is unsupported List Q: maintains unsupported (x i,a) pairs S(x j,a) : all values in other variable domains supported by (x j,a) In each step AC-4: –Picks and removes unsupported value from Q –Updates counters of potentially affected values

28. ©2007 Tarik Hadzic28 AC-4 (  )

29. ©2007 Tarik Hadzic29 AC-4 (  )

30. ©2007 Tarik Hadzic30 Example: AC-4 (  ) 2, 5 2,4 x z y List = {(z,5)} M = {} counter(z,2,y) = 2 counter(z,5,y) = 0 counter(z,2,x) = 1 counter(z,5,x) = 1 counter(x,2,z) = 1 counter(x,5,z) = 1 counter(y,2,z) = 1 counter(y,4,z) = 1

31. ©2007 Tarik Hadzic31 Example: AC-4 (  ) 2 2,5 2,4 x z y List = {} M = {(z,5)} counter(z,2,y) = 2 counter(z,5,y) = 0 counter(z,2,x) = 1 counter(z,5,x) = 1 counter(x,2,z) = 1 counter(x,5,z) = 1 counter(y,2,z) = 1 counter(y,4,z) = 1 for each (x j,a j ) in S (z,5) counter(x j,a j,z)-- if counter(x j,a j,x i )=0 then add (x j,a j ) to List S(z,5) = (x,5)

32. ©2007 Tarik Hadzic32 Example: AC-4 (  ) 2 2,5 2,4 x z y List = {} M = {(z,5)} counter(z,2,y) = 2 counter(z,5,y) = 0 counter(z,2,x) = 1 counter(z,5,x) = 1 counter(x,2,z) = 1 counter(x,5,z) = 0 counter(y,2,z) = 1 counter(y,4,z) = 1

33. ©2007 Tarik Hadzic33 Example: AC-4 (  ) 2 2, 5 2,4 x z y List = {(x,5)} M = {(z,5)} counter(z,2,y) = 2 counter(z,5,y) = 0 counter(z,2,x) = 1 counter(z,5,x) = 1 counter(x,2,z) = 1 counter(x,5,z) = 0 counter(y,2,z) = 1 counter(y,4,z) = 1

34. ©2007 Tarik Hadzic34 Example: AC-4 (  ) 2 2 2,4 x z y List = {} M = {(x,5),(z,5)} counter(z,2,y) = 2 counter(z,5,y) = 0 counter(z,2,x) = 1 counter(z,5,x) = 1 counter(x,2,z) = 1 counter(x,5,z) = 0 counter(y,2,z) = 1 counter(y,4,z) = 1 for each (x j,a j ) in S (x,5) counter(x j,a j,x)-- if counter(x j,a j,x i )=0 then add (x j,a j ) to List S(x,5) = (z,5)

35. ©2007 Tarik Hadzic35 Example: AC-4 (  ) 2 2 2,4 x z y List = {} M = {(x,5),(z,5)} counter(z,2,y) = 2 counter(z,5,y) = 0 counter(z,2,x) = 1 counter(z,5,x) = 1 counter(x,2,z) = 1 counter(x,5,z) = 0 counter(y,2,z) = 1 counter(y,4,z) = 1 S(x,5) = (z,5) (z,5) already in M, and not processed Nothing is added to List and AC-4 terminates!

36. ©2007 Tarik Hadzic36 AC-4 (  ) Time complexity O(ek 2 ) Space complexity O(ek 2 )

37. ©2007 Tarik Hadzic37 Directional Arc Consistency

38. ©2007 Tarik Hadzic38 Motivation Determining amount of inference for backtrack-free guarantee Restrict inference relative to a given ordering Arc consistency unnecessary when solution generated along a fixed ordering

39. ©2007 Tarik Hadzic39 Directional Arc-Consistency A network is directional arc-consistent relative to order d=(x 1,…,x n ) iff every arc (x i,x j ) is arc-consistent whenever i < j in the ordering DAC( ) for i=n to 1 by -1 for each j<i such that C ij  C Revise((x j ),x i )

40. ©2007 Tarik Hadzic40 Example, X={x 1,x 2,x 3,x 4 } D 1 ={red,white,black}, D 2 ={green,white,black}, D 3 ={red,white,blue}, D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 d = (x 1,x 2,x 3,x 4 ) x4x4 x3x3 x2x2 x1x1

41. ©2007 Tarik Hadzic41 Example x4x4 x3x3 x2x2 x1x1 D 1 ={red,white,black} D 2 ={green,white,black} D 3 ={red,white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4

42. ©2007 Tarik Hadzic42 Example D 1 ={red,white,black} D 2 ={green,white,black} D 3 ={red,white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 x4x4 x3x3 x2x2 x1x1 Revise((x 3 ),x 4 )

43. ©2007 Tarik Hadzic43 Example D 1 ={red,white,black} D 2 ={green,white,black} D 3 ={ red,white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 x4x4 x3x3 x2x2 x1x1 Delete red from D 3 !

44. ©2007 Tarik Hadzic44 Example D 1 ={red,white,black} D 2 ={green,white,black} D 3 ={white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 x4x4 x3x3 x2x2 x1x1 Revise((x 1 ),x 3 )

45. ©2007 Tarik Hadzic45 Example D 1 ={ red,white, black } D 2 ={green,white,black} D 3 ={white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 x4x4 x3x3 x2x2 x1x1 Delete red, black from D 1 !

46. ©2007 Tarik Hadzic46 Example D 1 ={white} D 2 ={green,white,black} D 3 ={white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 x4x4 x3x3 x2x2 x1x1 Revise((x 1 ),x 2 )

47. ©2007 Tarik Hadzic47 Example D 1 ={white} D 2 ={green,white,black} D 3 ={white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 x4x4 x3x3 x2x2 x1x1 No effect!

48. ©2007 Tarik Hadzic48 Example D 1 ={white} D 2 ={green,white,black} D 3 ={white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 x4x4 x3x3 x2x2 x1x1 DAC terminates!

49. ©2007 Tarik Hadzic49 Example D 1 ={white} D 2 ={green,white,black} D 3 ={white,blue} D 4 ={white,blue,black} R 12 : x 1 = x 2 R 13 : x 1 = x 3 R 34 : x 3 = x 4 Backtrack-free assignment: 1. x 1 = white 2. x 2 = white 3. x 3 = white 4. x 4 = white

50. ©2007 Tarik Hadzic50 Directional Arc-Consistency DAC( ) for i=n to 1 by -1 for each j<i such that C ij  C Revise((x j ),x i ) Complexity: O(ek 2 )

51. ©2007 Tarik Hadzic51 Global Constraints and Generalized Arc-Consistency

52. ©2007 Tarik Hadzic52 Definition Given constraint problem and (n- ary) constraint C’  C, C’ = (R,S) Variable x is (generalized) arc-consistent relative to constraint (R,S) iff for every a  D x there exists a tuple in the domains of variables in S, t  R such that t[x] = a

53. ©2007 Tarik Hadzic53 Example D x = [0,15], D y = [0,15], D z = [0,15] x+y+z  15, z ≥ 13 D x ← [0,2], D y ← [0,2] Alldiff(x 1,…,x n )

54. ©2007 Tarik Hadzic54 Modern Solver Architecture D 1, D 2, …, D n C1C1 C2C2 C3C3 1.Each Global Constraint Prunes Domains (highest level of pruning is GAC) 2.Constraint Store ”notifies” affected global constraints Global Constraints Constraint Store (Domain Store)

55. ©2007 Tarik Hadzic55 Backtrack-free search A constraint network is backtrack-free relative to a given ordering d={x 1,…,x n } if for every i  n, every partial solution of (x 1,…,x n ) can be consistently extended to include x n+1

56. ©2007 Tarik Hadzic56 Width A B C A B C C B A d 1 =(C,B,A) w(A) = 2 w(B)=0 w(C)=0 w(d 1 ) = 2 d 2 =(A,B,C) w(A) = 0 w(B)=1 w(C)=1 w(d 1 ) = 1 w(G) = min(w(d i )) w(G) = 1

57. ©2007 Tarik Hadzic57 Induced Width A B C d 1 =(C,B,A) w(A) = 2 w(B)=0 w(C)=0 w(d 1 ) = 2 Process nodes bottom-up Connect parrents of a processed node

58. ©2007 Tarik Hadzic58 Induced Width A B C d 1 =(C,B,A) w(A) = 2 w(B)=0 w(C)=0 w(d 1 ) = 2 Process nodes bottom-up Connect parrents of a processed node

59. ©2007 Tarik Hadzic59 Induced Width A B C d 1 =(C,B,A) w(A) = 2 w(B)=0 w(C)=0 w(d 1 ) = 2 Process nodes bottom-up Connect parrents of a processed node

60. ©2007 Tarik Hadzic60 Induced Width A B C d 1 =(C,B,A) w(A) = 2 w(B)=0 w(C)=0 w(d 1 ) = 2 Process nodes bottom-up Connect parrents of a processed node

61. ©2007 Tarik Hadzic61 Induced Width A B C d 1 =(C,B,A) w(A) = 2 w(B)=1 w(C)=0 w(d 1 ) = 2 w*(d 1 )=2 Process nodes bottom-up Connect parrents of a processed node Induced graph (G*,d 1 ) of an ordered graph (G,d 1 ) Induced width of (G,d 1 ) w*(d 1 )=2 Induced width of G is: w*=min(w*(d i )) G*