1. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Adam Smith Algorithm Design and Analysis L ECTURE 8 Greedy Graph Alg’s II Implementing Dijkstra MST

2. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Rough algorithm (Dijkstra) Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. Initialize S = { s }, d(s) = 0. Repeatedly choose unexplored node v which minimizes add v to S, and set d(v) =  (v). shortest path to some u in explored part, followed by a single edge (u, v) s v u d(u) S (e)

3. Review Question Is Dijsktra’s algorithm correct with negative edge weights? 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

4. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Proof of Correctness (Greedy Stays Ahead) Invariant. For each node u  S, d(u) is the length of the shortest path from s to u. Proof: (by induction on |S|) Base case: |S| = 1 is trivial. Inductive hypothesis: Assume for |S| = k  1. –Let v be next node added to S, and let (u,v) be the chosen edge. –The shortest s-u path plus (u,v) is an s-v path of length  (v). –Consider any s-v path P. We'll see that it's no shorter than  (v). –Let (x,y) be the first edge in P that leaves S, and let P' be the subpath to x. –P + (x,y) has length · d(x)+ (x,y) ·  (y) ·  (v) S s y v x P u P'

5. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Implementation For unexplored nodes, maintain –Next node to explore = node with minimum  (v). –When exploring v, for each edge e = (v,w), update Efficient implementation: Maintain a priority queue of unexplored nodes, prioritized by  (v). Priority Queue

6. Priority queues Maintain a set of items with priorities (= “keys”) –Example: jobs to be performed Operations: –Insert –Increase key –Decrease key –Extract-min: find and remove item with least key Common data structure: heap –Time: O(log n) per operation 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

7. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Pseudocode for Dijkstra(G, ) d[s]  0 for each v  V – {s} do d[v]   [v]   S   Q  V ⊳ Q is a priority queue maintaining V – S, keyed on  [v] while Q   do u  E XTRACT -M IN (Q) S  S  {u}; d[u]  [u] for each v  Adjacency-list[u] do if  [v] >  [u] + (u, v) then  [v]  d[u] + (u, v) explore edges leaving v Implicit D ECREASE -K EY

8. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Analysis of Dijkstra degree(u) times n times Handshaking Lemma  · m implicit D ECREASE -K EY ’s. while Q   do u  E XTRACT -M IN (Q) S  S  {u} for each v  Adj[u] do if  [v] >  [u] + w(u, v) then  [v]   [u] + w(u, v) † Individual ops are amortized bounds PQ Operation ExtractMin DecreaseKey Binary heap log n Fib heap † log n 1 Array n 1 Totalm log nm + n log nn2n2 Dijkstra n m d-way Heap HW3 m log m/n n

9. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Minimum spanning tree (MST) Input: A connected undirected graph G = (V, E) with weight function w : E  R. For now, assume all edge weights are distinct.. Output: A spanning tree T — a tree that connects all vertices — of minimum weight:

10. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Example of MST 612 5 14 3 8 10 15 9 7

11. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Example of MST 612 5 14 3 8 10 15 9 7

12. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Greedy Algorithms for MST Kruskal's: Start with T = . Consider edges in ascending order of weights. Insert edge e in T unless doing so would create a cycle. Reverse-Delete: Start with T = E. Consider edges in descending order of weights. Delete edge e from T unless doing so would disconnect T. Prim's: Start with some root node s. Grow a tree T from s outward. At each step, add to T the cheapest edge e with exactly one endpoint in T.

13. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Cycles and Cuts Cycle: Set of edges the form (a,b),(b,c),(c,d),…,(y,z),(z,a). Cut: a subset of nodes S. The corresponding cutset D is the subset of edges with exactly one endpoint in S. Cycle C = (1,2),(2,3),(3,4),(4,5),(5,6),(6,1) 1 3 8 2 6 7 4 5 Cut S = { 4, 5, 8 } Cutset D = (5,6), (5,7), (3,4), (3,5), (7,8) 1 3 8 2 6 7 4 5 S

14. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Cycle-Cut Intersection Claim. A cycle and a cutset intersect in an even number of edges. Proof: A cycle has to leave and enter the cut the same number of times. S V - S C

15. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Cut and Cycle Properties Cut property. Let S be a subset of nodes. Let e be the min weight edge with exactly one endpoint in S. Then the MST contains e. Cycle property. Let C be a cycle, and let f be the max weight edge in C. Then the MST does not contain f. f C S e is in the MST e f is not in the MST

16. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Proof of Cut Property Cut property: Let S be a subset of nodes. Let e be the min weight edge with exactly one endpoint in S. Then the MST T* contains e. Proof: (exchange argument) –Suppose e does not belong to T*. –Adding e to T* creates a cycle C in T*. –Edge e is both in the cycle C and in the cutset D corresponding to S  there exists another edge, say f, that is in both C and D. –T' = T*  { e } - { f } is also a spanning tree. –Since c e < c f, cost(T') < cost(T*). Contradiction. ▪ f T* e S

17. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Proof of Cycle Property Cycle property: Let C be a cycle in G. Let f be the max weight edge in C. Then the MST T* does not contain f. Proof: (exchange argument) –Suppose f belongs to T*. –Deleting f from T* creates a cut S in T*. –Edge f is both in the cycle C and in the cutset D corresponding to S  there exists another edge, say e, that is in both C and D. –T' = T*  { e } - { f } is also a spanning tree. –Since c e < c f, cost(T') < cost(T*). Contradiction. ▪ f T* e S

18. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Greedy Algorithms for MST Kruskal's: Start with T = . Consider edges in ascending order of weights. Insert edge e in T unless doing so would create a cycle. Reverse-Delete: Start with T = E. Consider edges in descending order of weights. Delete edge e from T unless doing so would disconnect T. Prim's: Start with some root node s. Grow a tree T from s outward. At each step, add to T the cheapest edge e with exactly one endpoint in T.

19. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Prim's Algorithm: Correctness Prim's algorithm. [Jarník 1930, Prim 1959] –Apply cut property to S. –When edge weights are distinct, every edge that is added must be in the MST –Thus, Prim’s alg. outputs the MST S

20. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Correctness of Kruskal [Kruskal, 1956]: Consider edges in ascending order of weight. –Case 1: If adding e to T creates a cycle, discard e according to cycle property. –Case 2: Otherwise, insert e = (u, v) into T according to cut property where S = set of nodes in u's connected component. Case 1 e v u Case 2 e S

21. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Review Questions Let G be a connected undirected graph with distinct edge weights. Answer true or false: Let e be the cheapest edge in G. Some MST of G contains e? Let e be the most expensive edge in G. No MST of G contains e?

22. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Review Questions Let G be a connected undirected graph with distinct edge weights. Answer true or false: Let e be the cheapest edge in G. Some MST of G contains e? (Answer: Yes, by the Cut Property ) Let e be the most expensive edge in G. No MST of G contains e? (Answer: False. Counterexample: if G is a tree, all its edges are in the MST)

23. Non-distinct edges? 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

24. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Implementation of Prim(G,w) I DEA : Maintain V – S as a priority queue Q (as in Dijkstra). Key each vertex in Q with the weight of the least- weight edge connecting it to a vertex in S. Q  V key[v]   for all v  V key[s]  0 for some arbitrary s  V while Q   dou  E XTRACT -M IN (Q) for each v  Adjacency-list[u] do if v  Q and w(u, v) < key[v] then key[v]  w(u, v) ⊳ D ECREASE -K EY  [v]  u At the end, {(v,  [v])} forms the MST.

25. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Handshaking Lemma   (m) implicit D ECREASE -K EY ’s. Q  V key[v]   for all v  V key[s]  0 for some arbitrary s  V while Q   dou  E XTRACT -M IN (Q) for each v  Adj[u] do ifv  Q and w(u, v) < key[v] then key[v]  w(u, v)  [v]  u Analysis of Prim degree(u) times n times  (n) total Time: as in Dijkstra

26. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Handshaking Lemma   (m) implicit D ECREASE -K EY ’s. while Q   dou  E XTRACT -M IN (Q) for each v  Adj[u] do ifv  Q and w(u, v) < key[v] then key[v]  w(u, v)  [v]  u Analysis of Prim degree(u) times n times † Individual ops are amortized bounds PQ Operation ExtractMin DecreaseKey Binary heap log n Fib heap † log n 1 Array n 1 Totalm log nm + n log nn2n2 Prim n m d-way Heap HW3 m log m/n n

27. 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Greedy Algorithms for MST Kruskal's: Start with T = . Consider edges in ascending order of weights. Insert edge e in T unless doing so would create a cycle. Reverse-Delete: Start with T = E. Consider edges in descending order of weights. Delete edge e from T unless doing so would disconnect T. Prim's: Start with some root node s. Grow a tree T from s outward. At each step, add to T the cheapest edge e with exactly one endpoint in T.

28. Union-Find Data Structures Operation\ ImplementationArray + linked-lists and sizes Balanced Trees Find (worst-case)(1)(log n) Union of sets A,B (worst-case) (min(|A|,|B|) (could be as large as (n) (log n) Amortized analysis: k unions and k finds, starting from singletons (k log k) 9/15/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne With modifications, amortized time for tree structure is O(n Ack(n)), where Ack(n), the Ackerman function grows much more slowly than log n. See KT Chapter 4.6