1. 242-535 ADA: 10. MSTs1 Objective o look at two algorithms for finding mimimum spanning trees (MSTs) over graphs Prim's algorithm, Kruskal's algorithm Algorithm Design and Analysis (ADA) 242-535, Semester 1 2014-2015 10. Minimum Spanning Trees (MSTs) 10. Minimum Spanning Trees (MSTs)

2. 242-535 ADA: 10. MSTs2 1.Minimum Spanning Tree 2.Prim's Algorithm 3.Kruskal's Algorithm 4.Difference between Prim and KruskalOverview

3. 242-535 ADA: 10. MSTs3 A minimum spanning tree T is a subgraph of a weighted graph G which contains all the verticies of G and whose edges have the minimum summed weight. Example weighted graph G: 1. Minimum Spanning Tree A B C D E F 3 4 5 1 63 2 6 2

4. 242-535 ADA: 10. MSTs4 A minimum spanning tree (weight = 12): A non-minimum spanning tree (weight = 20): A B C D E F 3 4 5 1 63 2 2 6 A B C D E F 3 4 5 1 63 2 6 2

5. 242-535 ADA: 10. MSTs5 Typical MST Applications

6. 242-535 ADA: 10. MSTs6 MSTs have the optimal substructure property: an optimal tree is composed of optimal subtrees. This can be seen by considering how to break a MST into parts: o Let T be an MST of G with an edge (u,v) o Removing (u,v) splits T into two trees T 1 and T 2 o Claim: T 1 is an MST of G 1 = (V 1,E 1 ), and T 2 is an MST of G 2 = (V 2,E 2 ) ( V 1 and V 2 do not share any vertices o Proof: w(T) = w(u,v) + w(T 1 ) + w(T 2 ) There can’t be a better tree than T 1 or T 2, or T would be suboptimal MST Optimality

7. 242-535 ADA: 10. MSTs7 Both dynamic programming (DP) and greedy algorithms can be used on problems that exhibit optimality. What's the difference between the two approaches? DP: use when the problem is optimal and there are repeating sub-problems Greedy: use when the problem is optimal, and each sub-problem can be solved without combining/examining smaller sub-sub-problems o this is called the greedy-choice property Optimality and Coding

8. 242-535 ADA: 10. MSTs8 DP solves sub-problems bottom-up since the current sub-problem may depend on sub-sub-problems. Greedy algorithms usually execute top-down, since a sub-problem can be solved without using sub-sub- problem solutions. In a greedy algorithm, we make whatever choice seems best at the moment and then solve any other sub-problems arising after the choice is made.

9. 242-535 ADA: 10. MSTs9 Since a MST is an optimal data structure, it is possible to use dynamic programming (DP) techniques o e.g. memoization, bottom-up execution In fact, both the MST algorithms in this part are greedy o at each iteration the choice of how to grow the MST only depends on the current state of the MST, not earlier sub- states or simpler versions DP, Greed and MST

10. 242-535 ADA: 10. MSTs10 Prim's algorithm finds a minimum spanning tree T by iteratively adding edges to T. At each iteration, a minimum-weight edge is added that does not create a cycle in the current T. The new edge must be connected to a vertex which is already in T. The algorithm can stop after |V|-1 edges have been added to T. 2. Prim's Algorithm

11. 242-535 ADA: 10. MSTs11 Tree prim(Graph G, int numVerts) { Tree T = anyVert(G); for i = 1 to numVerts-1 { Edge e = select an edge of minimum weight that is connected to a vertex already in T and does not form a cycle in T; T = T + e ; } return T } Simple Pseudocode

12. 242-535 ADA: 10. MSTs12 For the graph G. 1) Add any vertex to T o e.g A,T = {A} 2) Examine all the edges leaving {A} and add the vertex with the smallest weight. o edgeweight (A,B) 4 (A,C) 2 (A,E) 3 o add edge (A,C), T becomes {A,C} Informal Algorithm continued A B C D E F 3 4 5 1 63 2 6 2

13. 242-535 ADA: 10. MSTs13 3) Examine all the edges leaving {A,C} and add the vertex with the smallest weight. o edgeweightedgeweight (A,B) 4(C,D) 1 (A,E) 3(C,E) 6 (C,F) 3 o add edge (C,D), T becomes {A,C,D} continued

14. 242-535 ADA: 10. MSTs14 4) Examine all the edges leaving {A,C,D} and add the vertex with the smallest weight. o edgeweightedgeweight (A,B) 4(D,B) 5 (A,E) 3(C,E) 6 (C,F) 3(D,F) 6 o add edge (A,E) or (C,F), it does not matter o add edge (A,E), T becomes {A,C,D,E} continued

15. 242-535 ADA: 10. MSTs15 5) Examine all the edges leaving {A,C,D,E} and add the vertex with the smallest weight. o edgeweightedgeweight (A,B) 4(D,B) 5 (C,F) 3(D,F) 6 (E,F) 2 o add edge (E,F), T becomes {A,C,D,E,F} continued

16. 242-535 ADA: 10. MSTs16 6) Examine all the edges leaving {A,C,D,E,F} and add the vertex with the smallest weight. o edgeweightedgeweight (A,B) 4(D,B) 5 o add edge (A,B), T becomes {A,B,C,D,E,F} All the verticies of G are now in T, so we stop. continued

17. 242-535 ADA: 10. MSTs17 Resulting minimum spanning tree (weight = 12): A B C D E F 3 4 5 1 63 2 2 6

18. 242-535 ADA: 10. MSTs18 Build up the MST (black tree) by repeatedly adding the minimum crossing edge (thick red line) to it o a crossing edge connects a node in the MST to the rest of the graph. Ignore edges that form cycles (grey lines). Prim's Algorithm Graphically

19. 242-535 ADA: 10. MSTs19 The following weighted graph: Example 2 16 26 38 28 58 29 32 34 36 37 93 40 52 35 17 19

20. 242-535 ADA: 10. MSTs20 Building the MST 1 1 2 2 3 3

21. 242-535 ADA: 10. MSTs21 4 4 5 5 6 6

22. 242-535 ADA: 10. MSTs22 7 7 8 8 Finished

23. 242-535 ADA: 10. MSTs23 boolean marked[]; // for storing the vertices in the MST Queue mst; // for storing the edges in the MST MinPriQueue pq; // for storing the crossing (and ineligible) edges Prim's in More Detail

24. 242-535 ADA: 10. MSTs24 void prims(Graph graph, int start) { visit(graph, start); while (!pq.isEmpty()) { Edge e = pq.remove(); // get lowest weight edge int v = e.either(); // (v,w) are the ends of the edge e int w = e.other(v); if (!marked[v] || !marked[w]) { mst.add(e); // add edge to MST queue if (!marked[v]) // visit vertex v or w visit(graph, v); if (!marked[w]) visit(graph, w); } } // end of prims()

25. 242-535 ADA: 10. MSTs25 void visit(Graph graph, int v) /* Mark v and add to priority queue all the edges from v to unmarked vertices */ { marked[v] = true; for (Edge e : graph.adj(v)) if (!marked[e.other(v)]) pq.add(e); }

26. 242-535 ADA: 10. MSTs26 The running time depends on the implementation of the minimum priority queue, which uses a minimum heap. o An add() takes time O(log n), and remove is O(1) o At most E edges are added to the priority queue and at most E are removed. The algorithm has a worst case running time of O(E log E). Running Time

27. 242-535 ADA: 10. MSTs27 Krukal's algorithm finds a minimum spanning tree T by iteratively adding edges to T. At each iteration, a minimum-weight edge is added that does not create a cycle in the current T. The new edge can come from anywhere in G. The algorithm can stop after |V|-1 edges have been added to T. 3. Kruskal's Algorithm

28. 242-535 ADA: 10. MSTs28 Tree kruskal(Graph G, int numVerts) { Tree T = allVerts(G); for i = 1 to numVerts-1 { Edge e = select an edge of minimum weight from G which does not form a cycle in T; T = T + e ; } return T } Simple Pseudocode

29. 242-535 ADA: 10. MSTs29 Graph G: Example 1 a bc d e f g h i j k l 231 3 4 3 42 1 3 2 3 4 3 31 3 iteredge 1(c,d) 2(k,l) 3(b,f) 4(c,g) 5(a,b) 6(f, j) 7(b,c) 8(j,k) 9(g,h) 10(i, j) 11(a,e) continued

30. 242-535 ADA: 10. MSTs30 Minimum spanning tree (weight = 24): a bc d e f g h i j k l 231 3 4 3 42 1 3 2 3 4 3 31 3

31. 242-535 ADA: 10. MSTs31 Example 2 next MST edge edges sorted by weight black edges are used in MST

32. 242-535 ADA: 10. MSTs32 Queue mst; // for storing the edges in the MST MinPriQueue pq; // for storing the all the edges from G UnionFind uf; /* used to maintain disjoint sets of vertices: one for vertices in the MST, and other sets for nodes outside */ Kruskal's in More Detail

33. 242-535 ADA: 10. MSTs33 void kruskal(Graph graph) { pq.addAll(graph.edges()); // add all edges to pri queue uf.makeSets(graph.vertices()); // create disjoint sets of V's while ((!pq.isEmpty()) && (mst.size() < graph.vertices().size()-1)) { Edge e = pq.remove(); // get lowest weight edge int v = e.either(); // (v,w) are the ends of the edge e int w = e.other(v); if (!uf.isConnected(v, w)) { // are v and w not in same set? // only one of v or w must be in the MST set uf.union(v, w); // combine v and w's sets mst.add(e); // add edge to mst } } // end of kruskal()

34. 242-535 ADA: 10. MSTs34 As with Prim's algorithm, the running time depends on the implementation of the minimum priority queue, which uses a minimum heap. The algorithm also uses a Union-find data structure o E find() and isConnected() calls, which are both O(log n) The algorithm has a worst case running time of O(E log E) – same as Prim's o in practice, Prim's algorithm is usually faster than Kruskal's Running Time

35. 242-535 ADA: 10. MSTs35 A disjoint-set data structure keeps track of a set of elements split into disjoint (non-overlapping) subsets. Union-find consists of two main operations: o find(): report which subset a particular element is in o union(): join two subsets into a single subset o others: makeSets(), isConnected(), etc.Union-find

36. 242-535 ADA: 10. MSTs36

37. 242-535 ADA: 10. MSTs37 Prim's algorithm chooses an edge that must be connected to a vertex in the minimum spanning tree T. Kruskal's algorithm chooses an edge from G that may or may not be connected to a vertex in T. 4. Difference between Prim and Kruskal