1. Introduction to Algorithms: Shortest Paths I

2. Introduction to Algorithms
Shortest Paths I
Properties of shortest paths
Dijkstra’s Algorithm
Correctness
Analysis
Breadth-First Search
CS Computer Science II

3. CS 421 - Computer Science II
Paths in Graphs
Consider a digraph G = (V, E) with edge-weight function w : E  ℛ.
The weight of path
p = v1  v2  …  vk
is defined to be
k 1
w( p)   w(vi , vi1) .
i1
CS Computer Science II

4. CS 421 - Computer Science II
Paths in Graphs
Example: p: 𝑣1 𝑣3 𝑣5
4 –2
–5 1 𝑣2 𝑣4
w(p) = 4 + (-2) + (-5) + 1
= -2
CS Computer Science II

5. Shortest Paths
A shortest path from u to v is a path of minimum weight from u to v. The shortest- path weight from u to v is defined as
(u, v) = min{w(p) : p is a path from u to v}.
Note: (u, v) = , if no path from u to v exists.
CS Computer Science II

6. Well-Definedness of Shortest Paths
If a graph G contains a negative-weight cycle, then some shortest paths may not exist. …
Example:
< 0 u v
Then, (u, v) = -.
CS Computer Science II

7. CS 421 - Computer Science II
Optimal Substructure
Theorem. A sub-path of a shortest path is a shortest path.
CS Computer Science II

8. CS 421 - Computer Science II
Optimal Substructure
Theorem. A sub-path of a shortest path is a shortest path.
Proof. Suppose that p is the shortest path between the vertices u and v. And that x and y lie on that shortest path. p: u x y v
CS Computer Science II

9. CS 421 - Computer Science II
Optimal Substructure
Assume there’s another sub-path between x and y with a lower weight. Then that path would sub-path of a shortest path between u and v. But p is the shortest path. Which is a contradiction. Therefore, a sub-path of a shortest path is also a shortest path. p: u x y v
CS Computer Science II

10. CS 421 - Computer Science II
Triangle Inequality
Theorem. For all u, v, x  V, we have
(u, v)  (u, x) + (x, v).
Proof.
(u, v) u v
(u, x)
(x, v) x
CS Computer Science II

11. Single-Source Shortest Paths
Problem. From a given source vertex s  V, find the shortest-path weights (s, v) for all v  V.
If all edge weights w(u, v) are non-negative, all shortest-path weights must exist. (May be ).
IDEA: Greedy.
Maintain a set S of vertices whose shortest- path distances from s are known.
At each step add to S the vertex v  V – S
whose distance estimate from s is minimal.
Update the distance estimates of vertices adjacent to v.
CS Computer Science II

12. CS 421 - Computer Science II
Dijkstra’s Algorithm
d[s]  0
for each v  V – {s}
do d[v]  
S  
Q  V
while Q  
⊳ Q is a priority queue with V – S
do u  EXTRACT-MIN(Q)
S  S  {u}
for each v  Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
CS Computer Science II

13. Dijkstra’s Algorithm d[s]  0 for each v  V – {s} do d[v]   S  
Q  V
while Q  
⊳ Q is a priority queue with V – S
do u  EXTRACT-MIN(Q)
S  S  {u}
for each v  Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
relaxation step
Implicit DECREASE-KEY

14. Example of Dijkstra’s Algorithm2
Graph with non-negative edge weights: B D 10 8 A
1 4
7 9 3 C E 2
CS Computer Science II

15. Example of Dijkstra’s Algorithm  2
Graph with non-negative edge weights: B D 10 8 A
1 4
7 9 3
Q: A B C D E  C  E 2
d:    
S: {}
CS Computer Science II

16. Example of Dijkstra’s Algorithm 
“A”  EXTRACT-MIN(Q): 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E  C  E 2
d:    
S: { A }
CS Computer Science II

17. Example of Dijkstra’s Algorithm10 
Relax all edges leaving A: 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C  E 2
d:  
S: { A }
CS Computer Science II

18. Example of Dijkstra’s Algorithm10 
“C”  EXTRACT-MIN(Q): 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C  E 2
d:  
S: { A, C }
CS Computer Science II

19. Example of Dijkstra’s Algorithm7 11
Relax all edges leaving C: 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C 5 E 2 d:
S: { A, C }
CS Computer Science II

20. Example of Dijkstra’s Algorithm7 11
“E”  EXTRACT-MIN(Q): 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C 5 E 2 d:
S: { A, C, E }
CS Computer Science II

21. Example of Dijkstra’s Algorithm7 11
Relax all edges leaving E: 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C 5 E 2 d:
S: { A, C, E }
CS Computer Science II

22. Example of Dijkstra’s Algorithm7 11
“B”  EXTRACT-MIN(Q): 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C 5 E 2 d:
S: { A, C, E, B }
CS Computer Science II

23. Example of Dijkstra’s Algorithm7 9
Relax all edges leaving B: 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C 5 E 2 d:
S: { A, C, E, B }
CS Computer Science II

24. Example of Dijkstra’s Algorithm7 9
“D”  EXTRACT-MIN(Q): 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C 5 E 2 d:
S: { A, C, E, B }
CS Computer Science II

25. Example of Dijkstra’s Algorithm7 9 2 B D 10 8 A
1 4
7 9 3
Q: A B C D E 3 C 5 E 2 d:
S: { A, C, E, B }
CS Computer Science II

26. Dijkstra’s Algorithm – Finding Shortest Path
Algorithm determines weight of shortest path between source vertex s but how find the actual shortest path?
Shortest Path Tree: union of all shortest paths. For each vertex v  V and the last edge into that vertex that was relaxed (u, v), the union of all of those edges is a shortest path tree.
Has the property that there’s a unique path between s and all v  V and it’s the shortest path.
CS Computer Science II

27. Analysis of Dijkstra’s Algorithm
d[s]  0
for each v  V – {s}
do d[v]  
S  
|V |
times
Q  V
while Q  
do u  EXTRACT-MIN(Q)
S  S  {u}
for each v  Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
CS Computer Science II

28. Analysis of Dijkstra’s Algorithm
while Q  
|V |
times
do u  EXTRACT-MIN(Q)
S  S  {u}
for each v  Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
degree(u) times
Handshaking Lemma  (E) implicit DECREASE-KEY’s.
CS Computer Science II

29. Analysis of Dijkstra’s Algorithm
while Q  
|V |
times
do u  EXTRACT-MIN(Q)
S  S  {u}
for each v  Adj[u]
do if d[v] > d[u] + w(u, v)
then d[v]  d[u] + w(u, v)
degree(u) times
Time = (V·TEXTRACT-MIN + E·TDECREASE-KEY)
Note: Same formula as in the analysis of Prim’s minimum spanning tree algorithm.
CS Computer Science II

30. Analysis of Dijkstra (cont'd)
Time = (V)·TEXTRACT-MIN + (E)·TDECREASE-KEY Q
Total
TEXTRACT-MIN TDECREASE-KEY
array
binary heap
Fibonacci heap
O(V) O(1)
O(lg V) O(lg V)
O(V2)
O((V+E )lg V)
O(lg V) amortized
O(1)
amortized
O(E + V lg V)
worst case
CS Computer Science II

31. CS 421 - Computer Science II
Unweighted Graphs
Suppose that w(u, v) = 1 for all (u, v)  E.
Can Dijkstra’s algorithm be improved?
Use a simple FIFO queue instead of a priority queue.
CS Computer Science II

32. CS 421 - Computer Science II
Unweighted Graphs
Breadth-first search
while Q  
do u  DEQUEUE(Q)
for each v  Adj[u]
do if d[v] = 
then d[v]  d[u] + 1 ENQUEUE(Q, v)
Analysis: Time = O(V + E).
CS Computer Science II

33. Example of Breadth-First Search
a f h d
b g
e i c Q:
CS Computer Science II

34. Example of Breadth-First Search
a f h d
b g
e i c c
Q: a
CS Computer Science II

35. Example of Breadth-First Search
a f h d
b g
e i c 1 1 c
1 1
Q: a b d
CS Computer Science II

36. Example of Breadth-First Search
a f h d
b g
e i c 1 1 2
1 2 2 2
Q: a b d c e
CS Computer Science II

37. Example of Breadth-First Search
a f h d
b g
e i c 1 1 2 2
2 2
Q: a b d c e
CS Computer Science II

38. Example of Breadth-First Search
a f h d
b g
e i c 1 1 2 2 2
Q: a b d c e
CS Computer Science II

39. Example of Breadth-First Search
a f h d
b g
e i c 1 3 1 2 2 3
3 3
Q: a b d c e g i
CS Computer Science II

40. Example of Breadth-First Search4
a f h d
b g
e i c 1 3 1 2 2 3
3 4
Q: a b d c e g i f
CS Computer Science II

41. Example of Breadth-First Search4 4
a f h d
b g
e i c 1 3 1 2 2 3
4 4
Q: a b d c e g i f h
CS Computer Science II

42. Example of Breadth-First Search4 4
a f h d
b g
e i c 1 3 1 2 2 3 4
Q: a b d c e g i f h
CS Computer Science II

43. Example of Breadth-First Search4 4
a f h d
b g
e i c 1 3 1 2 2 3
Q: a b d c e g i f h
CS Computer Science II

44. CS 421 - Computer Science II

45. Correctness Dijkstra’s — Part I
Lemma.
Initializing d[s]  0 and d[v]   for all v  V – {s} establishes d[v]  (s, v) for all v  V, and this invariant is maintained over any sequence of relaxation steps.
CS Computer Science II

46. Correctness Dijkstra’s — Part I
Proof. Suppose not. Let v be the first vertex for
which d[v] < (s, v), and let u be the vertex that caused d[v] to change: d[v] = d[u] + w(u, v).
Then,
d[v] < (s, v) supposition
 (s, u) + (u, v) triangle inequality
 (s,u) + w(u, v) sh. path  specific path
 d[u] + w(u, v) v is first violation
Contradiction.
CS Computer Science II

47. Correctness Dijkstra’s — Part II
Lemma.
Let u be v’s predecessor on a shortest
path from s to v.
Then, if d[u] = (s, u) and edge (u, v) is relaxed, we have d[v]  (s, v) after the relaxation.
CS Computer Science II

48. Correctness Dijkstra’s — Part II
Proof. Observe that (s, v) = (s, u) + w(u, v). Suppose that d[v] > (s, v) before the relaxation. (Otherwise, we’re done.)
Then, the test d[v] > d[u] + w(u, v) succeeds, because
d[v] > (s, v) = (s, u) + w(u, v) = d[u] + w(u, v),
and the algorithm sets
d[v] = d[u] + w(u, v) = (s, v).
CS Computer Science II

49. Correctness Dijkstra’s — Part III
Theorem. Dijkstra’s algorithm terminates with
d[v] = (s, v) for all v  V.
CS Computer Science II

50. Correctness Dijkstra’s — Part III
Proof. It suffices to show that d[v] = (s, v) for every v  V when v is added to S. Suppose u is the first vertex added to S for which d[u]  (s, u). Let y be the first vertex in V – S along a shortest path from s to u, and let x be its predecessor: u s x
S, just before adding u. y
CS Computer Science II

51. Correctness Dijkstra’s — Part IIIu s x y
Since u is the first vertex violating the claimed invariant, we have d[x] = (s, x). When x was added to S, the edge (x, y) was relaxed, which implies that d[y] = (s, y)  (s, u)  d[u]. But, d[u]  d[y] by our choice of u. Contradiction.
CS Computer Science II

52. CS 421 - Computer Science II
Correctness of BFS
while Q  
do u  DEQUEUE(Q)
for each v  Adj[u]
do if d[v] = 
then d[v]  d[u] + 1 ENQUEUE(Q, v)
Key idea:
The FIFO Q in breadth-first search mimics the priority queue Q in Dijkstra.
Invariant: v comes after u in Q implies that
d[v] = d[u] or d[v] = d[u] + 1.
CS Computer Science II