1. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 1 Up-Trees Review of the UNION-FIND ADT Straight implementation with Up-Trees Path compression Worst-case analysis Applications

2. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 2 (Review) A Set of Elements Let S be a set of elements. e.g, S = { a, b, c, d, e}

3. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 3 (Review) Equivalence Relation Let R be a binary relation on S. Then R is an equivalence relation on S iff R is reflexive on S. R is symmetric R is transitive Example. Let’s say that two positive integers x and y not equal to 1 have the same base if there exist three integers b, m, and n such that x = b m and y = b n. So 36 and 216 have the same base, because 36 = 6 2 and 216 = 6 3. The relation of having the same base is an equivalence relation.

4. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 4 (Review) Another Equivalence Relation Let S = { a, b, c, d, e} Let R = { (a,a), (b,b), (c,c), (d,d), (e,e) (a,b), (b,a), (a,c), (c,a), (b,c), (c,b), (d,e), (e,d) } R can be partitioned into two subsets, such that within each subset, all the elements are related in both directions. P = { {a, b, c}, {d, e}} The two subsets are called equivalence classes.

5. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 5 The FIND Operation Let’s assume that we have an equivalence relation R, and a set of induced equivalence classes {E 1, E 2, E 3,..., E k } Each E i will have a representative element. FIND(x) for x in S, returns the representative element for the equivalence class containing x.

6. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 6 FIND -- an Example Let S = { a, b, c, d, e} P = { {a, b, c}, {d, e}} E 1 = {a, b, c}, E 2 = {d, e} Let a and d be the representative elements of E 1 and E 2, respectively. FIND(c) -> a FIND(d) -> d FIND(e) -> d

7. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 7 The UNION Operation Let’s assume that we have an equivalence relation R, and a set of induced equivalence classes {E 1, E 2, E 3,..., E k } UNION( E i, E j ) changes the partition so that E i and E j have been merged into one subset. For example, after UNION(E 2, E 3 ) we have {E 1, E 2 U E 3,..., E k } Note: Each E i can be represented by its represent. element. So it’s OK to write UNION(a, d)

8. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 8 Time Complexity In the straight array method, FIND takes O(1) and UNION takes O(n) Now, we’ll see that by using Up-trees, we can make the time complexity practically O(n) for a full sequence of n FIND and n UNION operations.

9. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 9 Up-Trees An up-tree, is a tree in which each node except the root has a single pointer (or simulated pointer) that points to its parent. A forest of up-trees is a set of up-trees.

10. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 10 Up-Tree Example A forest of up-trees can represent the subsets for the UNION-FIND ADT. ih b fc g ed a { { a, d, e, g}, { c }, { f, g, h, i } }

11. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 11 FIND using Up-Trees To determine the representative element for x, start at the node for x and follow the links to the root of x’s up-tree. FIND(h) = f ih b f

12. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 12 UNION using Up-Trees To merge to subsets, make the root of one up- tree point to the root of the other. UNION(a, f) ih b f g ed a

13. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 13 Time Complexity using Basic Up-Trees FIND(x): O(height(representative(x)) = O(n) UNION(x,y): O(1)

14. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 14 Improving on the Basic Implementation Typically, UNION and FIND operations are executed in sequences of about n/2 UNION operations and  (n) FIND operations. Therefore: Do a little extra work each time a UNION or FIND is performed, in order to reduce the work done in the future.

15. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 15 Enhanced UNION UNION(x,y): try to make the smaller tree point to the root of the larger. Height method: Try to use the higher root as the new root. (Difficult to easily update heights when path compression is done with FIND; instead we can use ranks -- estimated heights.) Weight method: Try to use the root of the larger up-tree as the new root. Keep a count of nodes at the root of each up-tree, using a 1-bit flag in each node to tell whether it’s the root.

16. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 16 Enhanced UNION (Cont) Even if no path compression is used with FIND, the enhanced UNION method cuts the time for a FIND operation to O(log n), assuming that a sequence of UNION and FIND operations is performed starting with singleton subsets (each element in its own subset), because the worst case up-trees formed will be “bushy” with branching factor at least 2 almost everywhere.

17. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 17 Enhanced FIND FIND(x): While performing this operation, compress the path(s) of nodes encountered along the way so that they are closer to the root. a. Full path compression. After finding the root r, make another pass from x to r making each node along the way point to r. b. Path halving: Make every node along the path from x to r point to its grandparent. c. Path splitting: Make every other node along the path from x to r point to its grandparent.

18. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 18 Path Compression FIND(h). ih b f c f c i bh

19. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 19 Worst-Case Analysis of Running Time for UNION-FIND using Path-Compressed Up-Trees Main points: (a) A simple technique may require a complex analysis method. (b) Ackermann’s function has an application. (c) Amortized analysis is a novel way to account for the running time of an algorithm. (d) UNION-FIND with path-compressed up-trees is highly efficient.

20. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 20 Ackermann’s Function A(1, j) = 2 j A(i, 1) = A(i-1, 2) for i  2 A(i, j) = A(i-1, A(i, j-1)) for i  2, j  2 Example values: A(1,1) = 2; A(1,2) = 4; A(1, 3) = 8; A(2,1) = 4; A(2, 2) = 16; A(2, 3) = 65536; A(2, 4) = 2 65536.

21. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 21 Inverse Ackermann’s Function  (p,q) = min{ z  A ( z,  p/q  ) > log 2 q }, p  q  1, z  1 For all practical cases,  (p,q)  4. The “Single-variable inverse Ackermann’s function”:  (n) = log*n  (n) = min{ k | log (log (... log(n)... ) )  1} k

22. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 22 Theorem Any sequence of m UNION/FIND operations, where m is  (n) can be done in O(m log*n) time. In theory, this is not linear, but in practice it is, since log*n  5 for values of n that come up in practice. The UNION operations are done by rank. FIND operations use path compression.

23. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 23 The Argument for the Theorem Lemma 1: During the execution of a sequence of UNION operations, a node of rank r must have at least 2 r descendants, including itself. Lemma 2: The number of nodes of rank r is no more than n/2 r. Lemma 3: During execution, the ranks of nodes on a path from a leaf to the root increase monotonically.

24. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 24 Argument, continued: “Pennies”) Time for FIND(x) is proportional to: The number of nodes long the path from x to its root r. To find the total time for all the FIND operations in a sequences of UNIONs and FINDs, we’ll “pay a penny for each node visited.” There will be two kinds of pennies: American and Canadian.

25. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 25 Argument: Rank Groups Before we explain how to pay, we group the nodes into “rank groups.” A rank group is a group of consecutive rank values that will allow efficient bookkeeping in the proof. Group 0: Rank 0 Group 1: Rank 1 Group 2: Rank 2 Group 3: Rank 3, 4 Group 4: Rank 5 through 16 Group 5: Rank 17 through 2 16 Group 6: Rank 65537 through 2 65536 Group 7: (impractically) extremely large rank values Define G(r) = Number of the group containing rank r.

26. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 26 Argument: Payment Rules Pay 1 American penny for node v into “the kitty” if v is the root, or if the parent of v is the root, or if the parent of v is in a different rank group from v. Pay 1 Canadian penny into the vertex otherwise.

27. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 27 Argument: Counting Pennies Lemma 4: For any FIND(x) operation, the total number of pennies deposited (American and Canadian) is equal to the number of nodes on the path from x to its root r. Lemma 5: Since we are performing a sequence of m FIND operations, the number of American pennies deposited is at most m(G(n) + 2), because in any FIND operation, 2 American pennies are deposited for the root and its child, and from x to r the ranks increase monotonically and there are only G(n) rank groups offering only G(n) opportunities to deposit an American penny according to the rule.

28. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 28 Argument: Counting Canadian Pennies Whenever a Canadian coin is deposited into a node, it will get a new parent of higher rank by path compression. A node in rank group g > 0 can be moved no more than F(g) - F(g - 1) times, where F(g) = greatest rank in rank group g. After this many moves, all further payments for the node use American pennies. (Its parent must then be in a different rank group.) Lemma 6: The number of nodes N(g) in rank group g > 0 is no more than n/2 F(g-1). To verify this, sum up the maximum number of nodes possible in each rank for ranks in the group.

29. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 29 Counting Canadian Pennies (Cont) For all nodes in rank group g, the number of (Canadian) pennies deposited is no more than F(g) n / 2 F(g - 1) because there each vertex in rank group g can receive at most F(g) pennies while its parent remains in the same group. Lemma 7: The total number of Canadian pennies paid is not greater than: n  F(g) / 2 F(g-1) g = 1 G(n)

30. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 30 Total Pennies The total of American and Canadian pennies paid cannot exceed m(G(n) + 2) + n  F(g) / 2 F(g-1) Choose F and G so as to make the upper bound as tight as possible... F(0) = 0; F(i) = 2 F(i-1) G(n) = 1 +  log*n  g = 1 G(n)

31. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 31 Theorem Again The running time of m UNION and FIND operations is O(m log*n) Proof: The total of American pennies is O(m G(n)). The total of Canadian pennies is O( n  1) = O(n G(n)). m is  (n) Grand total is O(n log* n). (Proof adapted from Weiss: Data Structures and Analysis in C, Benjamin Cummings, 1993). g = 1 G(n)

32. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 32 Applying the UNION-FIND ADT to Constructing Minimum Spanning Trees for Graphs Given a connected undirected weighted graph G = (V, E), a minimum spanning tree is a subgraph T of G such that: T is a tree (connected and contains no cycles), T contains each v in V, The total weight of all edges in T is as low as possible for all such spanning trees.

33. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 33 Example Graph + Minimum Spanning Tree e a b d c f g 6 8 2 3 27 24 12 8 17 9 35 5

34. CSE 373, Copyright S. Tanimoto, 2001 Up-trees - 34 Kruskal’s Algorithm Initialize a UNION-FIND structure with each vertex v of V in a separate subset. Consider each edge e of E in order of increasing weight. For each edge (u, v), perform FIND(u) and FIND(v), and if they are in the same subset, go on to the next edge. Otherwise, add (u, v) to the spanning tree, and perform a UNION on the two subsets. Keep a count of the number of edges added. When n-1 edges have been added (where n is the number of vertices), halt.