1. Data Structures and Algorithms CMPSC 465
LECTURES 32-37
Depth-first Search
Topological Sorting
Dijkstra’s algorithm for Shortest Paths
Adam Smith
11/20/2018
A. Smith; based on slides by K. Wayne

2. The (Algorithm) Design Process
Work out the answer for some examples
Look for a general principle
Does it work on *all* your examples?
Write pseudocode
Test your algorithm by hand or computer
Prove your algorithm is always correct
Check running time
Be prepared to go back to step 1!
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

3. Writing algorithms Clear and unambiguous Homework pitfalls:
Test: You should be able to hand it to any student in the class, and have them convert it into working code.
Homework pitfalls:
remember to specify data structures
writing recursive algorithms: don’t confuse the recursive subroutine with the first call
label global variables clearly
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

4. Writing proofs Purpose Who is your audience?
Determine for yourself that algorithm is correct
Convince reader
Who is your audience?
Yourself
Your classmate
Not the TA/grader
Main goal: Find your own mistakes
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

5. DFS: setting up notation
Maintain a global counter time
Maintain for each vertex v
Two timestamps:
v.d = time first discovered
v.f = time when finished
“color”: v.color
white = unexplored
gray = in process
black = finished
Parent v.π in DFS tree
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

6. DFS pseudocode Note: recursive function different from first call…
Exercise: change code to set “parent” pointer?
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

7. DFS example T = tree edge (drawn in red)
F = forward edge (to a descendant in DFS forest)
B= back edge (to an ancestor in DFS forest)
C = cross edge (goes to a vertex that is neither ancestor nor descendant)
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

8. DFS with adjacency lists
Outer code runs once, takes time O(n) (not counting time to execute recursive calls)
Recursive calls:
Run once per vertex
time = O(degree(v))
Total: O(m+n)
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

9. DFS Edge Types Tree Forward Backward Cross
Classifying edges according to type gives info about graph structure
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

10. Review Question
Give an algorithm that takes a directed graph G and finds a cycle if one exists. (Hint: use DFS)
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

11. Topological Sort
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

12. Directed Acyclic Graphs
Def. An DAG is a directed graph that contains no directed cycles.
Ex. Precedence constraints: edge (vi, vj) means vi must precede vj.
Def. A topological order of a directed graph G = (V, E) is an ordering of its nodes as v1, v2, …, vn so that for every edge (vi, vj) we have i < j. v2 v3 v6 v5 v4 v1 v2 v3 v4 v5 v6 v7 v7 v1
a DAG
a topological ordering

13. Precedence Constraints
Precedence constraints. Edge (vi, vj) means task vi must occur before vj.
Applications.
Course prerequisite graph: course vi must be taken before vj.
Compilation: module vi must be compiled before vj. Pipeline of computing jobs: output of job vi needed to determine input of job vj.
Getting dressed
shirt
mittens
socks
boots
jacket
underwear
pants

14. Recall from book Every DAG has a topological order
If G graph has a topological order, then G is a DAG.
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

15. Review Suppose your run DFS on a DAG G=(V,E) True or false?
Sorting by discovery time gives a topological order
Sorting by finish time gives a topological order
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

16. Shortest Paths
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

17. Shortest Path Problem Input: Find: shortest directed path from s to t.
Directed graph G = (V, E).
Source node s, destination node t.
for each edge e, length (e) = length of e.
length path = sum of edge lengths
Find: shortest directed path from s to t. 2 23 3 9 s 18 14 2 6 6 30 4 19
Length of path (s,2,3,5,t) is = 50. 11 5 15 5 6 20 16 t 7 44
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

18. Rough algorithm (Dijkstra)
Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known.
Initialize S = { s }, d(s) = 0.
Repeatedly choose unexplored node v which minimizes
add v to S, and set d(v) = (v).
shortest path to some u in explored part, followed by a single edge (u, v)
(e) v
d(u) u S s
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

19. Rough algorithm (Dijkstra)
Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known.
Initialize S = { s }, d(s) = 0.
Repeatedly choose unexplored node v which minimizes
add v to S, and set d(v) = (v).
Invariant: d(u) is known for all vertices in S
shortest path to some u in explored part, followed by a single edge (u, v)
BFS with weighted edges
(e) v
d(u) u S s
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

20. Demo of Dijkstra’s Algorithm2
Graph with nonnegative edge lengths: B D 10 8 A 1 4 7 9 3 C E 2
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

21. Demo of Dijkstra’s Algorithm
Initialize: 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2
S: {}
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

22. Demo of Dijkstra’s Algorithm
“A”  EXTRACT-MIN(Q): 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2
S: { A }
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

23. Demo of Dijkstra’s Algorithm10
Explore edges leaving A: 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2 3 10 3
S: { A }
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

24. Demo of Dijkstra’s Algorithm10
“C”  EXTRACT-MIN(Q): 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2 3 10 3
S: { A, C }
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

25. Demo of Dijkstra’s Algorithm7 11
Explore edges leaving C: 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2 3 5 10 3 7 11 5
S: { A, C }
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

26. Demo of Dijkstra’s Algorithm7 11
“E”  EXTRACT-MIN(Q): 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2 3 5 10 3 7 11 5
S: { A, C, E }
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

27. Demo of Dijkstra’s Algorithm7 11
Explore edges leaving E: 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2 3 5 10 3 7 11 5
S: { A, C, E } 7 11
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

28. Demo of Dijkstra’s Algorithm7 11
“B”  EXTRACT-MIN(Q): 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2 3 5 10 3 7 11 5
S: { A, C, E, B } 7 11
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

29. Demo of Dijkstra’s Algorithm7 9
Explore edges leaving B: 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2 3 5 10 3 7 11 5
S: { A, C, E, B } 7 11 9
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

30. Demo of Dijkstra’s Algorithm7 9
“D”  EXTRACT-MIN(Q): 2 B D 10 8 A 1 4 7 9 3 C E Q: A B C D E 2 3 5 10 3 7 11 5
S: { A, C, E, B, D } 7 11 9
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

31. Implementation For unexplored nodes, maintain
Next node to explore = node with minimum (v).
When exploring v, for each edge e = (v,w), update
Efficient implementation: Maintain a priority queue of unexplored nodes, prioritized by (v).
Priority Queue
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

32. Pseudocode for Dijkstra(G, )
d[s]  0
for each v Î V – {s}
do d[v]  ¥; p[v]  ¥
S  
Q  V ⊳ Q is a priority queue maintaining V – S, keyed on [v]
while Q ¹ 
do u  EXTRACT-MIN(Q)
S  S È {u}; d[u]  p[u]
for each v Î Adjacency-list[u]
do if [v] > [u] + (u, v)
then [v]  d[u] + (u, v)
explore edges leaving v
Implicit DECREASE-KEY
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

33. Analysis of Dijkstra while Q ¹  do u  EXTRACT-MIN(Q) S  S È {u}
for each v Î Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
n times
degree(u)
times
Handshaking Lemma  ·m implicit DECREASE-KEY’s.
PQ Operation
Dijkstra
Array
Binary heap
d-way Heap
Fib heap †
ExtractMin n n
log n
HW3
log n
DecreaseKey m 1
log n
HW3 1
Total n2
m log n
m log m/n n
m + n log n
† Individual ops are amortized bounds
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

34. Proof of Correctness (Greedy Stays Ahead)
Invariant. For each node u  S, d(u) is the length of the shortest path from s to u.
Proof: (by induction on |S|)
Base case: |S| = 1 is trivial.
Inductive hypothesis: Assume for |S| = k  1.
Let v be next node added to S, and let (u,v) be the chosen edge.
The shortest s-u path plus (u,v) is an s-v path of length (v).
Consider any s-v path P. We'll see that it's no shorter than (v).
Let (x,y) be the first edge in P that leaves S, and let P' be the subpath to x.
P + (x,y) has length · d(x)+ (x,y)· (y)· (v) P P' x y s S u v
Consider any path P from s to v. It is already at least as long as (v) by the time it leaves S.
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

35. Proof of Correctness More formally, we are proving two statements.
Invariant: At the end of each phase of the algorithm:
For all vertices reachable in one step from S: Pi(v) = min_{u in S} pi(u) + l(u,v)
For all vertices in S: pi(v) = d(s,v)
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

36. Review Question
Is Dijsktra’s algorithm correct with negative edge weights?
9/10/10
A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne