1. Tree Decomposition methods Chapter 9 ICS-275 Spring 2007

2. Graph concepts reviews: Hyper graphs and dual graphs A hypergraph is H = (V,S), V= {v_1,..,v_n} and a set of subsets Hyperegdes: S={S_1,..., S_l }. Dual graphs of a hypergaph: The nodes are the hyperedges and a pair of nodes is connected if they share vertices in V. The arc is labeled by the shared vertices. A primal graph of a hypergraph H = (V,S) has $V$ as its nodes, and any two nodes are connected by an arc if they appear in the same hyperedge. if all the constraints of a network R are binary, then its hypergraph is identical to its primal graph.

3. Acyclic Networks The running intersection property (connectedness): An arc can be removed from the dual graph if the variables labeling the arcs are shared along an alternative path between the two endpoints. Join graph: An arc subgraph of the dual graph that satisfies the connectedness property. Join-tree: a join-graph with no cycles Hypertree: A hypergraph whose dual graph has a join-tree. Acyclic network: is one whose hypergraph is a hypertree.

4. Solving acyclic networks Algorithm acyclic-solving applies a tree algorithm to the join-tree. It applies (a little more than) directional relational arc-consistency from leaves to root. Complexity: acyclic-solving is O(r l log l) steps, where r is the number of constraints and l bounds the number of tuples in each constraint relation (It assumes that join of two relations when one’s scope is a subset of the other can be done in linear time)

5. Example Constraints are: R_{ABC} = R_{AEF} = R_{CDE} = {(0,0,1) (0,1,0)(1,0,0)} and R_{ACE} = { (1,1,0) (0,1,1) (1,0,1) }. d= (R_{ACE},R_{CDE},R_{AEF},R_{ABC}). When processing R_{ABC}, its parent relation is R_{ACE}; we generate \pi_{ACE} (R_{ACE} x R_{ABC}), yielding R_{ACE} = {(0,1,1)(1,0,1)}. processing R_{AEF} we generate relation R_{ACE} = \pi_{ACE} ( R_{ACE} x R_{AEF} ) = {(0,1,1)}. processing R_{CDE} we generate: R_{ACE} = \pi_{ACE} ( R_{ACE} x R_{CDE} ) = {(0,1,1)}. A solution is generated by picking the only allowed tuple for R_{ACE}, A=0,C=1,E=1, extending it with a value for D that satisfies R_{CDE}, which is only D=0, and then similarly extending the assignment to F=0 and B=0, to satisfy R_{AEF} and R_{ABC}.

6. Recognizing acyclic networks Dual-based recognition: perform maximal spanning tree over the dual graph and check connectedness of the resulting tree. Dual-acyclicity complexity is O(e^3) Primal-based recognition: Theorem (Maier 83): A hypergraph has a join-tree iff its primal graph is chordal and conformal. A chordal primal graph is conformal relative to a constraint hypergraph iff there is a one-to-one mapping between maximal cliques and scopes of constraints.

7. Primal-based recognition Check cordality using max-cardinality ordering. Test conformality Create a join-tree: connect every clicque to an earlier clique sharing maximal number of variables

8. Tree-based clustering Convert a constraint problem to an acyclic-one: group subset of constraints to clusters until we get an acyclic problem. Hypertree embedding of a hypergraph H = (X,H) is a hypertree S = (X, S) s.t., for every h in H there is h_1 in S s.t. h is included in h_1. This yield algorithm join-tree clustering Hypertree partitioning of a hypergraph H = (X,H) is a hypertree S = (X, S) s.t., for every h in H there is h_1 in S s.t. h is included in h_1 and X is the union of scopes in h_1.

9. Join-tree clustering Input: A constraint problem R =(X,D,C) and its primal graph G = (X,E). Output: An equivalent acyclic constraint problem and its join-tree: T= (X,D, {C ‘}) 1. Select an d = (x_1,...,x_n) 2. Triangulation(create the induced graph along $d$ and call it G^*: ) for j=n to 1 by -1 do E  E U {(i,k)| (i,j) in E,(k,j) in E } 3. Create a join-tree of the induced graph G^*: a. Identify all maximal cliques (each variable and its parents is a clique). Let C_1,...,C_t be all such cliques, b. Create a tree-structure T over the cliques: Connect each C_{i} to a C_{j} (j < I) with whom it shares largest subset of variables. 4. Place each input constraint in one clique containing its scope, and let P_i be the constraint subproblem associated with C_i. 5. Solve P_i and let {R'}_i $ be its set of solutions. 6. Return C' = {R'}_1,..., {R'}_t the new set of constraints and their join-tree, T. Theorem: join-tree clustering trnasforms a constraint network into an acyclic network

10. Example of tree-clustering

11. Complexity of JTC THEOREM 5 (complexity of JTC) join-tree clustering is O(r k^ (w*(d)+1)) time and O(nk^(w*(d)+1)) space, where k is the maximum domain size and w*(d) is the induced width of the ordered graph. The complexity of acyclic-solving is O(nw*(d) log k k^(w*(d)+1))

12. Let P= be an automated reasoning problem. A tree decomposition is, such that T=(V,E) is a tree  associates a set of variables  (v)  X with each node  associates a set of functions  (v)  C with each node such that 1.  R i  C, there is exactly one v such that R i  (v) and scope(R i )  (v). 2.  x  X, the set {v  V|x  (v)} induces a connected subtree. Unifying tree-decompositions

13. Let P= be an automated reasoning problem. A tree decomposition is, such that T=(V,E) is a tree  associates a set of variables  (v)  X with each node  associates a set of functions  (v)  C with each node such that 1.  R i  C, there is exactly one v such that R i  (v) and scope(R i )  (v). 1a.  v,  (v)  scope(  (v)). 2.  x  X, the set {v  V|x  (v)} induces a connected subtree. HyperTree Decomposition w (tree-width) is max v  V |  (v)| hw (hypertree width) is maxv  V |  (v)| sep (max separator size) is max (u,v) |sep(u,v)|

14. Example of two join-trees again hyperTree- decomposition Tree decomposition

15. Cluster Tree Elimination Cluster Tree Elimination (CTE) works by passing messages along a tree-decomposition Basic idea: Each node sends one message to each of its neighbors Node u sends a message to its neighbor v only when u received messages from all its other neighbors

16. Constraint Propagation u v x1x1 x2x2 xnxn m(u,v)

17. Implementation tradeoffs Implementing Equation 1: The particular implementation of equation (1) in CTE can vary. One option is to generate the combined relation (1Ri2clusterv(u) Ri) before sending messages to neighbor v. The other option, which we assume here, is that the message sent to each neighbor is created without recording the relation (1Ri2clusterv(u) Ri). Rather, each tuple in the join is projected on the separator immediately after being created. This will yields a better memory utilization. Furthermore, when u sends a message to v its cluster may contain the message it received from v. Thus in a synchronized message passing we can allow a single enumeration of the tuples in cluster(u) when the messages are sent back towards the leaves, each of which be projected in parallel on the separators of the outgoing messages. The output of CTE is the original tree-decomposition where

18. Cluster-Tree Elimination (CTE)... m (v 1,u) m (v 2,u) m (v i,u)  (u)  (u) m (u,w) =  sep(u,w) [  j {m (v j,u) }   (u)]

19. Cluster Tree Elimination - properties Correctness and completeness: Algorithm CTE is correct, i.e. it computes the exact joint probability of every single variable and the evidence. Time complexity: O ( deg  (n+N)  d w*+1 ) Space complexity: O ( N  d sep ) wheredeg = the maximum degree of a node n = number of variables (= number of CPTs) N = number of nodes in the tree decomposition d = the maximum domain size of a variable w* = the induced width sep = the separator size Time and space by hyperwidth only if hypertree decomposition: time and O(N t^w*) space,

20. Cluster Tree Elimination - properties Correctness and completeness: Algorithm CTE is correct, i.e. it computes the exact joint probability of a single variable and the evidence. Time complexity: O ( deg  (r+N)  k w*+1 ) Space complexity: O ( N  d sep ) wheredeg = the maximum degree of a node nr= number of of CPTs N = number of nodes in the tree decomposition k= the maximum domain size of a variable w* = the induced width sep = the separator size JTC is O ( r  k w*+1 ) time and space

21. Example of CTE message propagation

22. Each function in a cluster Satisfy running intersection property Tree-width: number of variables in a cluster-1 Equals induced- width G E F CD B A A B C R(a), R(b,a), R(c,a,b) B C D F R(d,b), R(f,c,d) B E F R(e,b,f) E F G R(g,e,f) 2 4 1 3 EF BC BF Join-Tree Decomposition (Dechter and Pearl 1989)

23. A B C R(a), R(b,a), R(c,a,b) B C D F R(d,b), R(f,c,d),h (1,2) (b,c) B E F R(e,b,f), h (2,3) (b,f) E F G R(g,e,f) 2 4 1 3 EF BC BF sep(2,3)={B,F} elim(2,3)={C,D} Cluster Tree Elimination joinproject

24. ABC 2 4 1 3 BEF EFG EF BF BC BCDF Time: O ( exp(w*+1 )) Space: O ( exp(sep)) Time: O(exp(hw)) ( Gottlob et. Al., 2000 ) CTE: Cluster Tree Elimination G E F CD B A

25. Decomposable subproblems DEFINITION 8 (decomposable subproblem) Given a constraint problem R = (X;D;C) and a subset of variables Y µ X, a subproblem over Y, RY = (Y;DY ;CY ), is decomposable relative to R iff sol(RY ) = ¼Y sol(R) where sol(R) is the set of all solutions of network R

26. A 1 2 3 AC 12 32 AB 12 13 21 23 31 32 BCF 123 321 ABD 123 132 213 231 312 321 DFG 123 213 Distributed relational arc-consistency example A BC DF G The message that R2 sends to R1 is R1 updates its relation and domains and sends messages to neighbors

27. Distributed Arc-Consistency b) Constraint network DR-AC can be applied to the dual problem of any constraint network. A ABAC ABDBCF DFG A AB A A A B C B D F A

28. A 1 2 3 AC 12 32 AB 12 13 21 23 31 32 BCF 123 321 ABD 123 132 213 231 312 321 DFG 123 213 A AC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 DR-AC on a dual join-graph

29. A 1 2 3 AC 12 32 AB 12 13 21 23 31 32 BCF 123 321 ABD 123 132 213 231 312 321 DFG 123 213 A 1 2 3 A 1 2 3 A 1 2 3 A 1 3 B 1 2 3 1 3 B A 1 2 3 D 1 2 F 1 3 D 1 2 3 C 2 B 1 2 3 C 2 F 1 3 A 1 2 3 B 1 2 3 Iteration 1 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1

30. A 1 3 AC 12 32 AB 13 21 23 31 BCF 123 321 ABD 132 231 312 321 DFG 213 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 Iteration 1

31. A 1 3 AC 12 32 AB 13 21 23 31 BCF 123 321 ABD 132 231 312 321 DFG 213 A 1 3 A 1 3 A 1 2 3 A 1 3 A 1 3 D 2 F 1 3 D 1 2 C 2 B 1 3 C 2 F 1 A 1 2 3 B 1 3 B 1 2 3 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 Iteration 2 1 3 B

32. A 1 3 AC 12 32 AB 13 31 BCF 321 ABD 132 312 DFG 213 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1

33. A 1 3 A 1 3 A 1 3 B 3 A 1 3 D 2 F 1 D 2 C 2 B 1 3 C 2 F 1 A 1 3 A 1 3 A 1 3 AC 12 32 AB 13 31 BCF 321 ABD 132 312 DFG 213 B 1 3 B 1 3 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 Iteration 3

34. A 1 3 AC 12 32 AB 13 BCF 321 ABD 132 312 DFG 213 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 Iteration 3

35. A 1 3 A 1 3 A 1 B 3 A 1 3 D 2 F 1 D 2 C 2 B 3 C 2 F 1 A 1 3 A 1 3 A 1 3 AC 12 32 AB 13 BCF 321 ABD 132 312 DFG 213 B 1 3 B 1 3 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 Iteration 4

36. A 1 AC 12 32 AB 13 BCF 321 ABD 132 DFG 213 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 Iteration 4

37. A 1 A 1 A 1 B 3 B 3 B 3 A 1 D 2 F 1 D 2 C 2 B 3 C 2 F 1 A 1 A 1 A 1 AC 12 32 AB 13 BCF 321 ABD 132 DFG 213 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 Iteration 5

38. A 1 AC 12 AB 13 BCF 321 ABD 132 DFG 213 A ABAC ABDBCF DFG B 4 5 3 6 2 B D F A A A C 1 Iteration 5

39. Join-tree clustering is a restricted tree-decomposition: example

40. Adaptive-consistency as tree-decomposition Adaptive consistency is a message-passing along a bucket-tree Bucket trees: each bucket is a node and it is connected to a bucket to which its message is sent. The variables are the clicue of the triangulated graph The funcions are those placed in the initial partition

41. Bucket Elimination Adaptive Consistency (Dechter and Pearl, 1987) A E D C B E A D C B || R D BE, || R E || R DB || R DCB || R ACB || R AB RARA R C BE

42. From bucket-elimination to bucket-tree propagation

43. The bottom up messages

44. Adaptive-Tree-Consistency as tree-decomposition Adaptive consistency is a message-passing along a bucket-tree Bucket trees: each bucket is a node and it is connected to a bucket to which its message is sent. Theorem: A bucket-tree is a tree-decomposition Therefore, CTE adds a bottom-up message passing to bucket-elimination. The complexity of ATC is O(r deg k^(w*+1))

45. A ABDE FGI ABC BCE GHIJ CDEF FGH C H A C AABBC BE C C DECE F H F FGGHH GI A ABDE FGI ABC BCE GHIJ CDEF FGH C H A ABBC C DECE H F FGH GI ABCDE FGI BCE GHIJ CDEF FGH BC DECE F FGH GI ABCDE FGHIGHIJ CDEF CDE F GHI more accuracy less complexity Join-Graphs and Iterative Join-graph propagation (IJGP)