1. CSCI-256 Data Structures & Algorithm Analysis Lecture Note: Some slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved. 11

2. Shortest Path Problem Negative Cost Edges –Dijkstra’s algorithm assumes positive cost edges –For some applications, negative cost edges make sense –Shortest path not well defined if a graph has a negative cost cycle –Bellman-Ford algorithm finds shortest paths in a graph with negative cost edges (or reports the existence of a negative cost cycle). a b c s e g f 4 2 -3 6 4 -2 3 4 6 3 7 -4

3. Minimum Spanning Tree Minimum spanning tree: Given a connected graph G = (V, E) with real-valued edge weights c e, a MST is a subset of the edges T  E such that T is a spanning tree whose sum of edge weights is minimized 5 23 10 21 14 24 16 6 4 18 9 7 11 8 5 6 4 9 7 8 G = (V, E) T,  e  T c e = 50

4. Applications MST is a fundamental problem with diverse applications –Network design telephone, electrical, hydraulic, TV cable, computer, road –Approximation algorithms for NP-hard problems traveling salesperson problem, Steiner tree –Indirect applications max bottleneck paths LDPC codes for error correction image registration with Renyi entropy learning salient features for real-time face verification reducing data storage in sequencing amino acids in a protein model locality of particle interactions in turbulent fluid flows autoconfig protocol for Ethernet bridging to avoid cycles in a network –Cluster analysis

5. Greedy Algorithms Kruskal's algorithm –Start with T = . Consider edges in ascending order of cost. Insert edge e in T unless doing so would create a cycle Reverse-Delete algorithm –Start with T = E. Consider edges in descending order of cost. Delete edge e from T unless doing so would disconnect T Prim's algorithm –Start with some root node s and greedily grow a tree T from s outward. At each step, add the cheapest edge e to T that has exactly one endpoint in T Remark: All three algorithms produce a MST

6. Greedy Algorithm 1: Kruskal’s Algorithm Add the cheapest edge that joins disjoint components 9 2 13 6 4 11 5 7 20 14 15 10 1 8 12 16 22 17 3 t a e c g f b s u v Construct the MST with Kruskal’s algorithm Label the edges in order of insertion

7. Greedy Algorithm 2: Reverse-Delete Algorithm Delete the most expensive edge that does not disconnect the graph 9 2 13 6 4 11 5 7 20 14 15 10 1 8 12 16 22 17 3 t a e c g f b s u v Construct the MST with the reverse- delete algorithm Label the edges in order of removal

8. Greedy Algorithm 3: Prim’s Algorithm Extend a tree by including the cheapest out going edge 9 2 13 6 4 11 5 7 20 14 15 10 1 8 12 16 22 17 3 t a e c g f b s u v Construct the MST with Prim’s algorithm starting from vertex a Label the edges in order of insertion

9. Why do the greedy algorithms work? All these algorithms work by repeatedly inserting or deleting edges from a partial solution –Thus to analyze these algorithms, it would be useful to have in hand some basic facts saying when it is “safe” to include an edge in the MST or when it is “safe” to eliminate and edge on the grounds that it couldn’t possible be in the MST For simplicity, assume all edge costs are distinct. Thus, we can refer to “the MST”

10. When is it safe to include an edge in the MST? Edge inclusion lemma –Let S be a subset of V, and suppose e = (u, v) is the minimum cost edge of E, with u in S and v in V-S –e is in every MST of G. Or equivalently, if e is not in T, then T is not a MST SV - S e

11. Proof Suppose T is a spanning tree that does not contain e Add e to T, this creates a cycle The cycle must have some edge e 1 = (u 1, v 1 ) with u 1 in S and v 1 in V-S T 1 = T – {e 1 } + {e} is a spanning tree with lower cost Hence, T is not a minimum spanning tree SV - S e e1e1 e is the minimum cost edge between S and V-S

12. Optimality Proofs Prim’s Algorithm computes a MST Kruskal’s Algorithm computes a MST Show that when an edge is added to the MST by Prim or Kruskal, the edge is the minimum cost edge between S and V-S for some set S

13. Prim’s Algorithm S = { }; T = { }; while S != V choose the minimum cost edge e = (u,v), with u in S, and v in V-S add e to T add v to S

14. Prove Prim’s algorithm computes an MST Show an edge e is in the MST when it is added to T

15. Kruskal’s Algorithm Let C = {{v 1 }, {v 2 },..., {v n }}; T = { } while |C| > 1 Let e = (u, v) with u in C i and v in C j be the minimum cost edge joining distinct sets in C Replace C i and C j by C i U C j Add e to T

16. Prove Kruskal’s algorithm computes an MST Show an edge e is in the MST when it is added to T

17. When can we guarantee an edge is not in the MST? Cycle lemma –The most expensive edge on a cycle is never in a MST –Optimality of Reverse-Delete algorithm follows from this SV - S e e1e1 e is the most expensive edge on a cycle involving S and V-S

18. Dealing with the assumption of no equal weight edges Force the edge weights to be distinct –Add small quantities to the weights –Give a tie breaking rule for equal weight edges