1. Chapter 7: Greedy Algorithms
Chapter 7: Greedy Algorithms
7.4 Finding the Shortest Path Dijkstra’s Algorithm pp

2. Finding Shortest Paths
Finding Shortest Paths
Many real world situations can be represented using Graphs.
Many real world applications and problems require finding the shortest path in a Graph.
Consider the following “real world” example...

3. “Real World” Example 1
Ninja’s often find themselves in precarious situations where they must escape.
Like when they are being chased by a Pirate disguised as Johnny Depp.
A good Ninja always knows the shortest path to the escape route

4. Problem 1 Location of Ninja being chased by Pirate P I L F O A N U J D
Problem 1
Location of Ninja being chased by Pirate 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5
The only way out crazy Pirate maze.

5. Initially, the distance is set to infinite and all the paths are null.
A   B   C   D   E   F   G   I   J   L   N   O   P   T   U  
Keeps track of the shortest distance to each vertex and the path to get there.
Initially, the distance is set to infinite and all the paths are null. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

6. Initialization: Set the start vertex to zero.
A   B   C   D   E   F 0  G   I   J   L   N   O   P   T   U  
Initialization: Set the start vertex to zero.
Represents that you can get to F with zero cost. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

7. Iterate over v’s adjacency list, and update the distances.
A   B   C   D   E   F 0  G   I   J   L 1 F N 14 F O   P   T   U  
Step 1: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances.
Where can you go from F. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

8. Iterate over v’s adjacency list, and update the distances.
A 14 L B   C   D 13 L E   F 0  G   I 3 L J   L 1 F N 14 F O 7 L P   T   U  
Step 2: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances.
Where can you go from F and L. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

9. Iterate over v’s adjacency list, and update the distances.
A 14 L B   C   D 13 L E   F 0  G   I 3 L J   L 1 F N 14 F O 7 L P 4 I T   U  
Step 3: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances.
Where can you go from F, L, and I 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

10. Iterate over v’s adjacency list, and update the distances.
A 14 L B   C   D 13 L E   F 0  G   I 3 L J   L 1 F N 14 F O 6 P P 4 I T   U 8 P
Step 4: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances.
Where can you go from F, L, I and P 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

11. Iterate over v’s adjacency list, and update the distances.
A 14 L B   C   D 13 L E   F 0  G   I 3 L J   L 1 F N 14 F O 6 P P 4 I T   U 7 O
Step 5: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances.
Where can you go from F, L, I, P, and O. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

12. Iterate over v’s adjacency list, and update the distances.
A 14 L B   C   D 13 L E   F 0  G   I 3 L J 8 U L 1 F N 14 F O 6 P P 4 I T 9 U U 7 O
Step 6: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

13. Iterate over v’s adjacency list, and update the distances.
A 14 L B   C   D 11 J E   F 0  G   I 3 L J 8 U L 1 F N 14 F O 6 P P 4 I T 9 U U 7 O
Step 7: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

14. Iterate over v’s adjacency list, and update the distances.
A 14 L B   C   D 10 T E   F 0  G 12 T I 3 L J 8 U L 1 F N 14 F O 6 P P 4 I T 9 U U 7 O
Step 8: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

15. Iterate over v’s adjacency list, and update the distances.
A 12 D B 15 D C 17 D D 10 T E   F 0  G 12 T I 3 L J 8 U L 1 F N 14 F O 6 P P 4 I T 9 U U 7 O
Step 9: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

16. Iterate over v’s adjacency list, and update the distances.
A 12 D B 15 D C 17 D D 10 T E   F 0  G 12 T I 3 L J 8 U L 1 F N 14 F O 6 P P 4 I T 9 U U 7 O
Step 10: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

17. Iterate over v’s adjacency list, and update the distances.
A 12 D B 15 D C 17 D D 10 T E   F 0  G 12 T I 3 L J 8 U L 1 F N 13 A O 6 P P 4 I T 9 U U 7 O
Step 11: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

18. Iterate over v’s adjacency list, and update the distances.
A 12 D B 15 D C 15 N D 10 T E   F 0  G 12 T I 3 L J 8 U L 1 F N 13 A O 6 P P 4 I T 9 U U 7 O
Step 12: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

19. Iterate over v’s adjacency list, and update the distances.
A 12 D B 15 D C 15 N D 10 T E 16 C F 0  G 12 T I 3 L J 8 U L 1 F N 13 A O 6 P P 4 I T 9 U U 7 O
Step 13: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

20. Iterate over v’s adjacency list, and update the distances.
A 12 D B 15 D C 15 N D 10 T E 16 C F 0  G 12 T I 3 L J 8 U L 1 F N 13 A O 6 P P 4 I T 9 U U 7 O
Step 14: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

21. Iterate over v’s adjacency list, and update the distances.
A 12 D B 15 D C 15 N D 10 T E 16 C F 0  G 12 T I 3 L J 8 U L 1 F N 13 A O 6 P P 4 I T 9 U U 7 O
Step 15: Find the unmarked vertex with minimum distance (v). Mark it as visited
Iterate over v’s adjacency list, and update the distances. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

22. A 12 D B 15 D C 15 N D 10 T E 16 C F 0  G 12 T I 3 L J 8 U L 1 F N 13 A O 6 P P 4 I T 9 U U 7 O
Final Step: This data structure can be used to compute the shortest path from F to all other vertices. 1 2 1 P I L F 14 2 4 6 13 O A 1 N 4 12 1 5 2 2 U J D C 1 3 7 2 4 1 3 5 4 1 T G B E 3 4 5

23. A B C D E F G I J L N O P T U
L 14     3  1F   
I 14     3L   
P 14      I  8
O 14      P  7
U 14     O
J 14     U
T 14    U
D T 
G  12T
A 12D 
N  A
C N
B 15D
B C P I L F O A N U J D C T G B E 1 2 4 6 12 13 14 7 5 3

24. “Real World” Lesson Learned
Just do the Flip Out Dance to escape from the crazy Ninja maze with minimum damage.

25. Serious Lessons Learned
Dijkstra’s algorithm is greedy.
It always chooses the vertex with minimum distance from the source vertex.
Optimization after each step
Dijkstra’s algorithm updates the distances
Shorter paths are discovered as minimum distances get updated.

26. Worst Case
Finding the shortest path from a Start Vertex to a End Vertex could require marking every vertex.
The Crazy Pirate Maze example showed us this fact.
However, could we have stopped the algorithm after we marked the End Vertex as visited?

27. Example 2 F  0  P I L F O A N U J D C T G B E 1 9 1 5 2 4 9 13 1 412 1 5 2 5 U J D C 1 3 1 2 4 4 3 5 4 1 T G B E 3 4 5

28. Example 2 F  0  L  1 F N  5 F P I L F O A N U J D C T G B E 1 9 14 9 13 O A 1 N 4 12 1 5 2 5 U J D C 1 3 1 2 4 4 3 5 4 1 T G B E 3 4 5

29. Example 2 F  0  L  1 F N  5 F I  10 L O  10 L D  13 L A  14 L9 1 P I L F 5 2 4 9 13 O A 1 N 4 12 1 5 2 5 U J D C 1 3 1 2 4 4 3 5 4 1 T G B E 3 4 5

30. Example 2
F  0  L  1 F N  5 F A  6 N I  10 L O  10 L C  10 N D  13 L 1 9 1 P I L F 5 2 4 9 13 O A 1 N 4 12 1 5 2 5 U J D C 1 3 1 2 4 4 3 5 4 1 T G B E 3 4 5

31. Example 2
F  0  L  1 F N  5 F A  6 N D  8 A I  10 L O  10 L C  10 N 1 9 1 P I L F 5 2 4 9 13 O A 1 N 4 12 1 5 2 5 U J D C 1 3 1 2 4 4 3 5 4 1 T G B E 3 4 5

32. Example 2
F  0  L  1 F N  5 F A  6 N D  8 L C  9 D I  10 L O  10 L J  11 D G  11 D T  12 D B  13 D 1 9 1 P I L F 5 2 4 9 13 O A 1 N 4 12 1 5 2 5 U J D C 1 3 1 2 4 4 3 5 4 1 T G B E 3 4 5

33. Question: Is 9 the minimum distance from F to C?
F  0  L  1 F N  5 F A  6 N D  8 L C  9 D I  10 L I  10 L J  11 D G  11 D E  11 C T  12 D B  13 D 1 9 1 P I L F 5 2 4 9 13 O A 1 N 4 12 1 5 2 5 U J D C 1 3 1 2 4 4 3 5 4 1 T G B E 3 4 5

34. Question: How do we know there isn’t a shorter path?
F  0  L  1 F N  5 F A  6 N D  8 L C  9 D I  10 L I  10 L J  11 D G  11 D E  11 C T  12 D B  13 D 1 9 1 P I L F 5 2 4 9 13 O A 1 N 4 12 1 5 2 5 U J D C 1 3 1 2 4 4 3 5 4 1 T G B E 3 4 5

35. Technicality
The textbook’s version of Dijkstra’s algorithm stops when all vertices are marked.
which computes the shortest distance from a Start Vertex to all the other vertices.
But, if you are only interested in the shortest path from A to B, you can stop the algorithm as soon as B is marked.

36. Worst Case
Nonetheless, there are A to B problems that require all vertices to be marked.
Thus, Dijkstra’s Algorithm takes O(N) steps.
Now, in the worst case, how much work does one step take?

37. Worst Case Step
Dijkstra’s table will have N entries, one for each vertex.
You have to pick the unmarked entry with minimum distance.
Given an array of N numbers, how long does it take to find the minimum?
What would be the overall worst case running time of Dijkstra’s algorithm consider it has to find a minimum N times?
A B C D E F G I J L N O P T U
14    3   I  8
Find minimum

38. Worst Case Step N Steps Find minimum for each step O(N2) so far...
Worst Case Step
N Steps
Find minimum for each step
O(N2)
so far...
A B C D E F G I J L N O P T U
14    3   I  8
Find minimum

39. Worst Case Step
Worse yet, once we find the closest vertex, we must update all the distances.
It is possible that the closest vertex could be adjacent to all vertices.
Thus, there could be up to N updates per step.
Does this impact the Big-O?

40. Worst Case Example A  0 B  440 C  900 D  900 E  900 F  900 F E A
Worst Case Example
A  0
B  440
C  900
D  900
E  900
F  900 F E
900
900 A
900 D
900
440 B C

41. Worst Case Example A  0 B  440 C  630 D  880 E  880 F  880 F E A
Worst Case Example
A  0
B  440
C  630
D  880
E  880
F  880 F E
900
900
440 A
900 D
440
440
900
440 B C
210

42. Worst Case Example A  0 B  440 C  630 D  670 E  730 F  730 F E A
Worst Case Example
A  0
B  440
C  630
D  670
E  730
F  730 F E
100
900
100
900
440 A
900 D
440
440
900
440 40 B C
210

43. Worst Case Example A  0 B  440 C  630 D  670 E  680 F  690 F E A
Worst Case Example
A  0
B  440
C  630
D  670
E  680
F  690 F E
100 10
900
100
900 20
440 A
900 D
440
440
900
440 40 B C
210

44. Worst Case Example A  0 B  440 C  630 D  670 E  680 F  685 F E A
Worst Case Example
A  0
B  440
C  630
D  670
E  680
F  685 5 F E
100 10
900
100
900 20
440 A
900 D
440
440
900
440 40 B C
210

45. Worst Case Example
The Problem: An update requires finding a vertex in the table.
Find is O(N)
Each of the N2 updates required N operations. F A B C
900 E
440
210
100 40 20 10 5
A  0
B  440
C  630
D  670
E  680
F  685
Step 1: N-1 updates
Step 2: N-2 updates
Step 3: N-3 updates
...
Step i: N-i updates
Step N-1: 1 update
Step N: 0 updates D
Worst case on number of updates:

46. Worst Case Example Note:
Worst Case Example
Note:
The worst case can only arise in a complete graph with N(N-1)/2 edges.
The number of updates is really O(M), where M is the number of edges, which is bounded by O(N2) F A B C
900 E
440
210
100 40 20 10 5
A  0
B  440
C  630
D  670
E  680
F  685
Step 1: N-1 updates
Step 2: N-2 updates
Step 3: N-3 updates
...
Step i: N-i updates
Step N-1: 1 update
Step N: 0 updates D
Worst case on number of updates:

47. Worst Case Algorithm N * (N + (N-1)*N) = O(N3)
Worst Case Algorithm
while there are no un-marked vertices {
find the vertex v with minimum weight
mark it as visited
for all of v’s edges {
update the minimum distance and path to each vertex adjacent to v }
N vertices * Find min: O(N)
N-1 edges *
1 update: requires a find: O(N)
N * (N + (N-1)*N) = O(N3)

48. Data Structures to the Rescue
Is there a data structure we can use instead of an array.
Goals:
Find a vertex in constant time
Update a distance in constant time
Find minimum as efficiently as possible

49. Sorting & Binary Search: Sort the array by Distance
Sorting & Binary Search: Sort the array by Distance
while there are no un-marked vertices {
find the vertex v with minimum weight
mark it as visited
for all of v’s edges {
update the minimum distance and path to each vertex adjacent to v
} re-sort the vertices by distance }
N vertices * Find min: O(1)
N-1 edges *
1 update: requires a find: O(N) Sort: O(N log N)
N * (1 + (N-1)*N + N*log N) = O(N3)

50. Sorting & Binary Search: Sort the array by Vertex Label
Sorting & Binary Search: Sort the array by Vertex Label
while there are no un-marked vertices {
find the vertex v with minimum weight
mark it as visited
for all of v’s edges {
update the minimum distance and path to each vertex adjacent to v }
N vertices * Find min: O(N)
N-1 edges *
1 update: find: O(log N)
N * (N + (N-1)*log N) = O(N2 log N)

51. Hash Table: Constant time vertex lookup
Hash Table: Constant time vertex lookup
while there are no un-marked vertices {
find the vertex v with minimum weight
mark it as visited
for all of v’s edges {
update the minimum distance and path to each vertex adjacent to v }
N vertices * Find min: O(N)
N-1 edges *
1 update: find: O(1)
N * (N + (N-1)*1) = O(N2)

52. N * log N + M) = O(N log N + M)
Binary Heap with Hash Index: Constant time vertex lookup Find min: O(log N) Updates: O(1)
while there are no un-marked vertices {
find the vertex v with minimum weight
mark it as visited
for all of v’s edges {
update the minimum distance and path to each vertex adjacent to v }
N vertices * Find min: O(log N)
N-1 edges *
1 update: find: O(1)
N * log N + M) = O(N log N + M)

53. Is there a difference?

54. 3 2 8 P I L F 4 2 4 1 2 O A 1 N 4 12 6 9 6 2 U J D C 8 3 2 2 4 8 7 2 5 8 T G B E 5 4 1