1. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Adam Smith Algorithm Design and Analysis L ECTURE 7 Greedy Graph Algorithms Topological sort Shortest paths

2. The (Algorithm) Design Process 1.Work out the answer for some examples 2.Look for a general principle –Does it work on *all* your examples? 3.Write pseudocode 4.Test your algorithm by hand or computer –Does it work on *all* your examples? 5.Prove your algorithm is always correct 6.Check running time Be prepared to go back to step 1! 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

3. Writing algorithms Clear and unambiguous –Test: You should be able to hand it to any student in the class, and have them convert it into working code. Homework pitfalls: –remember to specify data structures –writing recursive algorithms: don’t confuse the recursive subroutine with the first call –label global variables clearly 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

4. Writing proofs Purpose –Determine for yourself that algorithm is correct –Convince reader Who is your audience? –Yourself –Your classmate –Not the TA/grader Main goal: Find your own mistakes 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

5. Exploring a graph Classic problem: Given vertices s,t ∈ V, is there a path from s to t? Idea: explore all vertices reachable from s Two basic techniques: Breadth-first search (BFS) Explore children in order of distance to start node Depth-first search (DFS) Recursively explore vertex’s children before exploring siblings 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne How to convert these descriptions to precise algorithms?

6. A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Adjacency-matrix representation The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1.. n, 1.. n] given by A[i, j] = 1if (i, j) ∈ E, 0if (i, j)  E. 2 2 1 1 3 3 4 4 A1234 1 2 3 4 0110 0010 0000 0010 Storage: (V 2 ) Good for dense graphs. Lookup: O(1) time List all neighbors: O(|V|) 9/10/10

7. Adjacency list representation An adjacency list of a vertex v ∈ V is the list Adj[v] of vertices adjacent to v. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne 2 2 1 1 3 3 4 4 Adj[1] = {2, 3} Adj[2] = {3} Adj[3] = {} Adj[4] = {3} For undirected graphs, | Adj[v] | = degree(v). For digraphs, | Adj[v] | = out-degree(v). How many entries in lists? 2|E| (“handshaking lemma”) Total (V + E) storage — good for sparse graphs. Typical notation: n = |V| = # vertices m = |E| = # edges

8. DFS pseudocode Maintain a global counter time Maintain for each vertex v –Two timestamps: v.d = time first discovered v.f = time when finished –“color”: v.color white = unexplored gray = in process black = finished –Parent v.π in DFS tree 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

9. DFS pseudocode Note: recursive function different from first call… Exercise: change code to set “parent” pointer? 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

10. DFS example T = tree edge (drawn in red) F = forward edge (to a descendant in DFS forest) B= back edge (to an ancestor in DFS forest) C = cross edge (goes to a vertex that is neither ancestor nor descendant) 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

11. DFS with adjacency lists Outer code runs once, takes time O(n) (not counting time to execute recursive calls) Recursive calls: –Run once per vertex –time = O(degree(v)) Total: O(m+n) 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

12. Toplogical Sort 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

13. Directed Acyclic Graphs Def. An DAG is a directed graph that contains no directed cycles. Ex. Precedence constraints: edge (v i, v j ) means v i must precede v j. Def. A topological order of a directed graph G = (V, E) is an ordering of its nodes as v 1, v 2, …, v n so that for every edge (v i, v j ) we have i < j. a DAG a topological ordering v2v2 v3v3 v6v6 v5v5 v4v4 v7v7 v1v1 v1v1 v2v2 v3v3 v4v4 v5v5 v6v6 v7v7

14. Precedence Constraints Precedence constraints. Edge (v i, v j ) means task v i must occur before v j. Applications. n Course prerequisite graph: course v i must be taken before v j. n Compilation: module v i must be compiled before v j. Pipeline of computing jobs: output of job v i needed to determine input of job v j. n Getting dressed underwear pants jacket shirt boots socks mittens

15. Recall from book Every DAG has a topological order If G graph has a topological order, then G is a DAG. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

16. Review Suppose your run DFS on a DAG G=(V,E) True or false? –Sorting by discovery time gives a topological order –Sorting by finish time gives a topological order 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

17. Shortest Paths 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

18. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Shortest Path Problem Input: –Directed graph G = (V, E). –Source node s, destination node t. –for each edge e, length (e) = length of e. –length path = sum of edge lengths Find: shortest directed path from s to t. Length of path (s,2,3,5,t) is 9 + 23 + 2 + 16 = 50. s 3 t 2 6 7 4 5 23 18 2 9 14 15 5 30 20 44 16 11 6 19 6

19. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Rough algorithm (Dijkstra) Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. Initialize S = { s }, d(s) = 0. Repeatedly choose unexplored node v which minimizes add v to S, and set d(v) =  (v). shortest path to some u in explored part, followed by a single edge (u, v) s v u d(u) S (e)

20. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Rough algorithm (Dijkstra) Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. Initialize S = { s }, d(s) = 0. Repeatedly choose unexplored node v which minimizes add v to S, and set d(v) =  (v). shortest path to some u in explored part, followed by a single edge (u, v) s v u d(u) S (e) BFS with weighted edges Invariant: d(u) is known for all vertices in S

21. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 Graph with nonnegative edge lengths:

22. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 Initialize: ABCDEQ: 0  S: {} 0   

23. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A } 0    “A”  E XTRACT -M IN (Q):

24. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A } 0 10 3    Explore edges leaving A: 

25. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A, C } 0 10 3    “C”  E XTRACT -M IN (Q): 

26. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A, C } 0 7 35 11 10  7115 Explore edges leaving C: 

27. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A, C, E } 0 7 35 11 10  7115 “E”  E XTRACT -M IN (Q): 

28. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A, C, E } 0 7 35 11 10  7115 7 Explore edges leaving E:

29. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A, C, E, B } 0 7 35 11 10  7115 7 “B”  E XTRACT -M IN (Q):

30. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A, C, E, B } 0 7 35 9 10  7115 7 Explore edges leaving B: 9

31. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Demo of Dijkstra’s Algorithm A BD CE 10 3 1479 8 2 2 ABCDEQ: 0  S: { A, C, E, B, D } 0 7 35 9 10  7115 7 9 “D”  E XTRACT -M IN (Q):

32. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Proof of Correctness (Greedy Stays Ahead) Invariant. For each node u  S, d(u) is the length of the shortest path from s to u. Proof: (by induction on |S|) Base case: |S| = 1 is trivial. Inductive hypothesis: Assume for |S| = k  1. –Let v be next node added to S, and let (u,v) be the chosen edge. –The shortest s-u path plus (u,v) is an s-v path of length  (v). –Consider any s-v path P. We'll see that it's no shorter than  (v). –Let (x,y) be the first edge in P that leaves S, and let P' be the subpath to x. –P + (x,y) has length · d(x)+ (x,y) ·  (y) ·  (v) S s y v x P u P'

33. Review Question Is Dijsktra’s algorithm correct with negative edge weights? 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

34. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Implementation For unexplored nodes, maintain –Next node to explore = node with minimum  (v). –When exploring v, for each edge e = (v,w), update Efficient implementation: Maintain a priority queue of unexplored nodes, prioritized by  (v). Priority Queue

35. Priority queues Maintain a set of items with priorities (= “keys”) –Example: jobs to be performed Operations: –Insert –Increase key –Decrease key –Extract-min: find and remove item with least key Common data structure: heap –Time: O(log n) per operation 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

36. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Pseudocode for Dijkstra(G, ) d[s]  0 for each v  V – {s} do d[v]   [v]   S   Q  V ⊳ Q is a priority queue maintaining V – S, keyed on  [v] while Q   do u  E XTRACT -M IN (Q) S  S  {u}; d[u]  [u] for each v  Adjacency-list[u] do if  [v] >  [u] + (u, v) then  [v]  d[u] + (u, v) explore edges leaving v Implicit D ECREASE -K EY

37. 9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Analysis of Dijkstra degree(u) times n times Handshaking Lemma  · m implicit D ECREASE -K EY ’s. while Q   do u  E XTRACT -M IN (Q) S  S  {u} for each v  Adj[u] do if d[v] > d[u] + w(u, v) then d[v]  d[u] + w(u, v) † Individual ops are amortized bounds PQ Operation ExtractMin DecreaseKey Binary heap log n Fib heap † log n 1 Array n 1 Totalm log nm + n log nn2n2 Dijkstra n m d-way Heap HW3 m log m/n n