Minimum Spanning Trees Featuring Disjoint Sets HKOI Training 2006 Liu Chi Man (cx) 25 Mar 2006

2 Prerequisites Asymptotic complexity Set theory Elementary graph theory Priority queues (or heaps)

3 Graphs A graph is a set of vertices and a set of edges G = (V, E) Number of vertices = |V| Number of edges = |E| We assume simple graph, so |E| = O(|V| 2 )

4 Roadmap What is a tree? Disjoint sets Minimum spanning trees Various tree topics

5 What is a Tree?

6 Trees in graph theory In graph theory, a tree is an acyclic, connected graph  Acyclic means “ without cycles ”

7 Properties of trees |E| = |V| - 1  |E| =  (|V|) Between any pair of vertices, there is a unique path Adding an edge between a pair of non-adjacent vertices creates exactly one cycle Removing an edge from the tree breaks the tree into two smaller trees

8 Definition? The following four conditions are equivalent:  G is connected and acyclic  G is connected and |E| = |V| - 1  G is acyclic and |E| = |V| - 1  Between any pair of vertices in G, there exists a unique path G is a tree if at least one of the above conditions is satisfied

9 Other properties of trees Bipartite Planar A tree with at least two vertices has at least two leaves (vertices of degree 1)

10 Roadmap What is a tree? Disjoint sets Minimum spanning trees Various tree topics

11 The Union-Find problem N balls initially, each ball in its own bag  Label the balls 1, 2, 3,..., N Two kinds of operations:  Pick two bags, put all balls in these bags into a new bag (Union)  Given a ball, find the bag containing it (Find)

12 The Union-Find problem An example with 4 balls Initial: {1}, {2}, {3}, {4} Union {1}, {3}  {1, 3}, {2}, {4} Find 3. Answer: {1, 3} Union {4}, {1,3}  {1, 3, 4}, {2} Find 2. Answer: {2} Find 1. Answer {1, 3, 4}

13 Disjoint sets Disjoint-set data structures can be used to solve the union-find problem Each bag has its own representative ball  {1, 3, 4} is represented by ball 3 (for example)  {2} is represented by ball 2

14 Implementation 1: Naive arrays Bag[x] := representative of the bag containing x  Union takes O(N) and Find takes O(1) Slight modifications give  U is the size of the union Worst case: O(MN) for M operations

15 Implementation 1: Naive arrays How to union Bag[x] and Bag[y]?  Z := Bag[x] For each ball v in Z do Bag[v] := Bag[y] Can I update the balls in Bag[y] instead? Rule: Update the balls in the smaller bag  O(MlgN) for M union operations

16 Implementation 2: Forest A forest is a collection of trees Each bag is represented by a rooted tree, with the root being the representative ball Example: Two bags --- {1, 3, 5} and {2, 4, 6, 7}.

17 Implementation 2: Forest Find(x)  Traverse from x up to the root Union(x, y)  Merge the two trees containing x and y

18 Implementation 2: Forest Initial: Union 1 3: Union 2 4: Find 4:

19 Implementation 2: Forest Union 1 4: Find 4:

20 Implementation 2: Forest How to represent the trees?  Leftmost-Child-Right-Sibling (LCRS)? Too complicated  Parent array Parent[x] := parent of x If x is a tree root, set Parent[x] := x

21 Implementation 2: Forest The worst case is still O(MN ) for M operations  What is the worst case? Improvements  Union-by-rank  Path compression

22 Union-by-rank We should avoid tall trees Root of the taller tree becomes the new root when union So, keep track of tree heights (ranks) Good Bad

23 Path compression See also the solution for Symbolic Links (HKOI2005 Senior Final) Find(x): traverse from x up to root Compress the x-to-root path at the same time

24 Path compression Find(4) The root is

25 U-by-rank + Path compression We ignore the effect of path compression on tree heights to simplify U-by-rank U-by-rank alone gives O(MlgN) U-by-rank + path compression gives O(M  (N))   : inverse Ackermann function  (N)  5 for practically large N

26 Roadmap What is a tree? Disjoint sets Minimum spanning trees Various tree topics

27 Minimum spanning trees Given a connected graph G = (V, E), a spanning tree of G is a graph T such that  T is a subgraph of G  T is a tree  T contains every vertex of G A connected graph must have at least one spanning tree

28 Minimum spanning trees Given a weighted connected graph G, a minimum spanning tree T* of G is a spanning tree of G with minimum total edge weight Application: Minimizing the total length of wires needed to connect up a collection of computers

29 Minimum spanning trees Two algorithms  Kruskal ’ s algorithm  Prim ’ s algorithm

30 Kruskal ’ s algorithm Choose edges in ascending weight greedily, while preventing cycles

31 Kruskal ’ s algorithm Algorithm  T is an empty set  Sort the edges in G by their weights  For (in ascending weight) each edge e do If T  {e} is acyclic then  Add e to T  Return T

32 Kruskal ’ s algorithm How to detect a cycle?  Depth-first search (DFS) O(V) per check O(VE) overall  Disjoint set Vertices are balls, connected components are bags

33 Kruskal ’ s algorithm Algorithm (using disjoint-set)  T is an empty set  Create bags {1}, {2}, …, {V}  Sort the edges in G by their weights  For (in ascending weight) each edge e do Suppose e connects vertices x and y If Find(x)  Find(y) then  Add e to T, then Union(Find(x), Find(y))  Return T

34 Kruskal ’ s algorithm The improved time complexity is O(ElgV) The bottleneck is sorting

35 Prim ’ s algorithm In Kruskal ’ s algorithm, the MST-in- progress scatters around Prim ’ s algorithm grows the MST from a “ seed ” Prim ’ s algorithm iteratively chooses the lightest grow-able edge  A grow-able edge connects a grown vertex and a non-grown vertex

36 Prim’s algorithm Algorithm  Let seed be any vertex, and Grown := {seed}  Initially T is an empty set  Repeat |V|-1 times Let e=(x,y) be the lightest grow-able edge Add e to T Add x and y to Grown  Return T

37 Prim’s algorithm How to find the lightest grow-able edge?  Check all (grown, non-grown) vertex pairs Too slow  Each non-grown vertex x keeps a value nearest[x], which is the weight of the lightest edge connecting x to some grown vertex Nearest[x] =  if no such edge

38 Prim’s algorithm How to use nearest?  Grow the vertex (x) with the minimum nearest-value Which edge? Keep track on it!  Since x has just been grown, we need to update the nearest-values of all non-grown vertices Only need to consider edges incident to x

39 Prim’s algorithm Try to program Prim’s algorithm You may find that it’s very similar to Dijkstra’s algorithm for finding shortest paths!  Almost only a one-line difference

40 Prim ’ s algorithm Per round...  Finding minimum nearest-value: O(V)  Updating nearest-values: O(V) (Overall O(E)) Overall: O(V 2 +E) = O(V 2 ) time Using a binary heap,  O(lgV) per Finding minimum  O(lgV) per Updating  Overall: O(ElgV) time

41 MST Extensions Second-best MST  We don ’ t want the best! Online MST  See IOI2003 Path Maintenance Minimum bottleneck spanning tree  The bottleneck of a spanning tree is the weight of its maximum weight edge  An algorithm that runs in O(V+E) exists

42 MST Extensions (NP-Hard) Minimum Steiner Tree  No need to connect all vertices, but at least a given subset B  V Degree-bounded MST  Every vertex of the spanning tree must have degree not greater than a given value K For a discussion of NP-hardness, please attend [Talk] Introduction to Complexity Theory on 3 June

43 Roadmap What is a tree? Disjoint sets Minimum spanning trees Various tree topics

44 Various tree topics (List) Center, eccentricity, radius, diameter Tree isomorphism  Canonical representation Pr ü fer code Lowest common ancestor (LCA) Counting spanning trees

45 Supplementary readings Advanced:  Disjoint set forest (Lecture slides) Disjoint set forest  Prim ’ s algorithm Prim ’ s algorithm  Kruskal ’ s algorithm Kruskal ’ s algorithm  Center and diameter Center and diameter Post-advanced (so-called Beginners):  Lowest common ancestor Lowest common ancestor  Maximum branching Maximum branching