More Chapter 7: Greedy Algorithms Kruskal’s Minimum Spanning Tree Algorithm.

Minimum Spanning Tree (MST) Problem Given a weighted graph, i.e. a graph with edge weights… try to find a sub-graph that (i) connects all the nodes and (ii) the sum of the edge weights is minimal. This sub-graph will always be a tree. ABC DEF G H I J

MST Example Given the following graph, find the MST First, we want all the nodes connected Second, we want to pick the lowest weight edges. ABC DEF G H I J

MST Example Greedy step 1: Start with A and select the minimum edge This would connect D. ABC DEF G H I J Total Weight: 3

MST Example Greedy step 2: From D select the minimum edge This would connect H. ABC DEF G H I J Total Weight: 6

MST Example Greedy step 3: From H select the minimum edge This would connect I. ABC DEF G H I J Total Weight: 11

MST Example Greedy step 4: From I select the minimum edge This would connect J. ABC DEF G H I J Total Weight: 16

MST Example Greedy step 5: From J select the minimum edge This would connect G. ABC DEF G H I J Total Weight: 20

MST Example Greedy step 6: From G select the minimum edge This would connect F. ABC DEF G H I J Total Weight: 24

MST Example Greedy step 6: From G select the minimum edge This would connect F. ABC DEF G H I J Total Weight: 24

MST Example What is the running time of the algorithm? ABC DEF G H I J Total Weight: 27

MST Example N steps Each step must find the minimum edge from a node Worst case: N-1 + N-1 + N-1 + … + N-1 = O(N 2 ) ABC DEF G H I J

MST Example You might think the worst case is: N-1 + N-2 + N-3 + … = O(N 2 ) because at every step you connect a node and don’t have to consider it’s edges. However, think about how the algorithm would actually be implemented, and how you would keep track of this info?

MST Example You might think the worst case is: N-1 + N-2 + N-3 + … = O(N 2 ) pick node v from the node_list. while node_list is not empty { mark v as visited. min_hop = infinity; foreach of v’s edges (v,w) if (w is not visited) if (edge (v,w) < min_hop) { min_hop = edge(v,w) min_edge = w; } remove v from node from node_list v = w; } The first while loop will always take N iterations The foreach loop could take N-1 iteration in a complete graph

MST Example Greedy step 8: From E select the minimum edge This would connect B. ABC DEF G H I J Total Weight: 33

MST Example Greedy step 9: From B select the minimum edge This would connect C. ABC DEF G H I J Total Weight: 38

MST Example This greedy algorithm failed Why? ABC DEF G H I J Total Weight: 38

MST Example It makes a local decision. From E, it chooses to go to B We have to consider other options. ABC DEF G H I J Total Weight: 38

Kruskal’s Algorithm Solves the Minimum Spanning Tree Problem using a better Greedy Approach Input: –List of edges in a graph –n – the number of vertices Output: –Prints the list of edges in the Minimum Spanning Tree

ABC DEF G HI J

Kruskal’s kruskal(e, n) { sort(e); ABC DEF G H I J

Kruskal’s kruskal(e, n) { sort(e); ABC DEF G H I J A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7

kruskal(e, n) { sort(e); for (i = A to J) makeset(i) ABC DEF G H I J A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ABCDEFGHIJ

kruskal(e, n) {... count = 0; i = 1 ABC DEF G H I J A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ABCDEFGHIJ Count 0 i1i1

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ABCDEFGHIJ count 0 i1i1 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ABCDHEFGIJ Count 1 i2i2 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ABCDHEFGIJ Count 2 i3i3 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ADHBCEFGIJ Count 3 i4i4 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ADHBCEFGIJ Count 4 i5i5 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ADHBCEFGIJ Count 5 i6i6 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ADHBCEFGIJ Count 6 i7i7 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ADHBCEFGJI Count 7 i8i8 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ADHBCEFGJI Count 8 i9i9 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ADHBCEFGJI Count 8 i 10 n 10

kruskal(e, n) { while (count < n-1) { if (findset(e[i].v) != findset(e[i].w)) { print(e[i].v + “ ”+ e[i].w); count++; union(e[i].v, e[i].w); } i++; } A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ADHBCEFGJI Count 9 i 11 n 10

A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ABC DEF G H I J

A D 3 D H 3 EF 3 AB 4 C F 4 F G 4 G J 4 A E 5 H I 5 I J 5 F J 5 BC 5 B E 6 C G 6 F I 6 DE 7 E H 7 ABC DEF G H I J

ABC DEF G H I J A B C DE F G H I J

Theorem pp. 280 Let G be a connected, weighted graph, and let G’ be a sub-graph of a minimal spanning tree of G. Let C be a component of G’, and let S be the set of all Edges with one vertex in C and the other not in C. If we add a minimum weight edge in S to G’, the resulting graph is also contained in a minimal spanning tree of G

Theorem pp. 280 Let G be a connected, weighted graph, and let G’ be a sub-graph of a minimal spanning tree of G. Let C be a component of G’, and let S be the set of all Edges with one vertex in C and the other not in C. If we add a minimum weight edge in S to G’, the resulting graph is also contained in a minimal spanning tree of G ABC DEF G H I J G A B C DE F G H I J Minimal Spanning Tree of G

Theorem pp. 280 G’ be a sub-graph of a minimal spanning tree of G. Let C be a component of G’, and let S be the set of all Edges with one vertex in C and the other not in C. ABC DEF G H I J G G’ Subset of Minimal Spanning Tree of G S C A DE AB 4 A E 5 DE 7 D H 3

Theorem pp. 280 If we add a minimum weight edge from S to G’, the resulting graph is also contained in a minimal spanning tree of G ABC DEF G H I J G G’ Subset of Minimal Spanning Tree of G S C A DE AB 4 A E 5 DE 7 D H 3

Theorem 7.2.6: Kruskal’s Algorithm finds minimum spanning tree Proof by induction G’ is a sub-graph constructed by Kruskal’s Algorithm G’ is initially empty but each step of the Algorithm increases the size of G’ Inductive Assumption: G’ is contained in the MST.

Theorem 7.2.6: Kruskal’s Algorithm finds minimum spanning tree Proof by induction Let (v,w) be the next edge selected by Kruskal’s Algorithm Kruskal’s algorithm finds the minimum weight edge (v,w) such that v and w are not already in G’ C can be any subset of the MST, so you can always construct a C such that v is in C and w is not. Therefore, by Theorem 7.2.5, when (v,w) is added to G’, the resulting graph is also contained in the MST.