Discussion #35 Chapter 7, Section 5.5 1/15 Discussion #35 Dijkstra’s Algorithm

Discussion #35 Chapter 7, Section 5.5 2/15 Topics Shortest path Greedy algorithms Dijkstra’s algorithm

Discussion #35 Chapter 7, Section 5.5 3/15 Shortest Path Minimum length path from one node to another (e.g. from a to e, the shortest path is 2) Algorithms –Breadth-First-Search, O(n 2 ) in the worst case –Floyd’s, O(n 3 ); but finds all –BFS, starting at each node, O(nm), which beats Floyd’s for sparse graphs b ca ed

Discussion #35 Chapter 7, Section 5.5 4/15 Shortest Path in Weighted Graphs Weights on edges: positive numbers Algorithms –Simple BFS no longer works (e.g. the shortest path from a to b is not the edge that connects them) –Floyd’s, O(n 3 ); finds all Is there an algorithm that beats Floyd’s? –for the single best path? –for all paths? b ca ed

Discussion #35 Chapter 7, Section 5.5 5/15 Dijkstra’s Algorithm Find the shortest weighted path between two given nodes. (Easier to find the minimum from one to all.) Intuitively, special BFS: choose smallest accumulated path length Algorithm b ca ed Initialize D (distance) table. 2.Until all nodes are settled, 2a.find smallest distance & settle node. 2b.adjust distances in D for unsettled nodes.

Discussion #35 Chapter 7, Section 5.5 6/15 Important Aside: Greedy Algorithms A greedy algorithm always takes the best immediate or local solution while finding an answer. Greedy algorithms find optimal solutions for some optimization problems, but may find (far) less- than-optimal solutions for other optimization problems. –Dijkstra’s algorithm is greedy. –There is no greedy algorithm for the traveling salesman problem (find the shortest path to visit all nodes). When greedy algorithms work, they are usually best.

Discussion #35 Chapter 7, Section 5.5 7/15 Trace of Dijkstra’s Algorithm all nodes settled a,d,c,e,b 44 a,d,c,e 243 adjust w/e a,d,c 2252 adjust w/c a  edcba settled distanceiteration b ca ed a,d  251 adjust w/d 2 1. Initialize D (start=a) 2. Until all nodes are settled: -find smallest distance & settle node. -adjust distances in D for unsettled nodes. Circled numbers: shortest path lengths from a.

Discussion #35 Chapter 7, Section 5.5 8/15 Trace with Different Weights all nodes settled a,d,e,c,b 54 a,d,e,c 363 adjust w/c a,d,e 2462 adjust w/e a  edcba settled distanceiteration b ca ed a,d  461 adjust w/d 2 1. Initialize D (start=a) 2. Until all nodes are settled: -find smallest distance & settle node. -adjust distances in D for unsettled nodes. 5 Circled numbers: shortest path lengths from a.

Discussion #35 Chapter 7, Section 5.5 9/15 Proof that Dijkstra’s Algorithm Works Loop Invariant: For each settled node x, the minimum distance from the start node s to x is D(x). Additional observations (which help us in the proof) are also loop invariants. –(a) The shortest path from s to any settled node v consists only of settled nodes. –(b) The shortest path from s to any unsettled node y is at least the largest settled difference so far.

Discussion #35 Chapter 7, Section /15 Proof by Induction (on the number of iterations) Basis: 0 iterations: x = s, D(s) = 0 (which is the minimum since all weights are positive) –For (a): initially s = v, the only settled node –For (b): initially all nodes except s are unsettled, and the shortest path to each is at least 0, indeed > 0. Induction: Assume that the main loop invariant and the loop invariants (a) and (b) hold for k iterations and show that they hold for k+1 iterations.

Discussion #35 Chapter 7, Section /15 Proof of Induction Part (Sketch) Assume that the main loop invariant does not hold for the k+1 st iteration, then there is a shorter path from s to x. k settled s... v  x k+1 settled s... v  x By the induction hypotheses, since Dijkstra’s algorithm chooses x, the path from s to x is the shortest using settled nodes, and the path from s to y is no shorter than s to x. Since all weights are positive, the path from y to x is at least 1, so that the assumed shorter path cannot exist. Further (a) and (b) continue to hold: (a) After settling x, the path from s to x consists of only settled nodes. (b) The adjustments are only for paths from x to any unsettled nodes. Since the distance from s to any unsettled node y is at least D(x), and D(x) is the largest settled distance so far. s... v  x... w  y Assume Dijkstra’s algorithm selects x to be settled.

D(x) 0a (b,5) (c,2) (d,1) 5b (a,5) (c,2) 2c (a,2) (b,2) (d,10) (e,1) 1d (a,1) (c,10) (e,1)  e (d,1) (c,1) Discussion #35 Chapter 7, Section /15 Big-Oh for Dijkstra’s Algorithm 1.O(n)  visit at most all edges for a node  can’t be more than n 2.O(n)  each node needs to be settled 2a.O(n)  look through D(x) list to find smallest 2b.O(k)  for chosen node, go through list of length k b ca ed Adjacency List: 1. Initialize D (start=a) 2. Until all nodes are settled: 2a. find smallest distance & settle node. 2b. adjust distances in D for unsettled nodes. { O(n 2 ) Over all iterations, the k’s add up to 2(m  # of edges in the initialization). dominates  Unroll the loop: n + k 1 + n + k 2 + … + n + k (n-1) = (n  1)n = O(n 2 ) = 2(m  # of edges in the initialization) = O(m)

Discussion #35 Chapter 7, Section /15 Big-Oh for Dijkstra’s Algorithm Clever using POTs: 1.O(nlogn)  POT (Partially Ordered Tree) construction dominates 2.O(n)  each node needs to be settled 2a.O(logn)  swap and bubble down 2b.O(klogn)  for chosen node, go through list of length k b ca ed Initialize D (start=a) 2. Until all nodes are settled: 2a. find smallest distance & settle node. 2b. adjust distances in D for unsettled nodes. { O(mlogn) dominates D(x) 0a (b,5) (c,2) (d,1) 5b (a,5) (c,2) 2c (a,2) (b,2) (d,10) (e,1) 1d (a,1) (c,10) (e,1)  e (d,1) (c,1) Adjacency List:  Over all iterations, the k’s add up to 2(m  # of edges in the initialization).

Discussion #35 Chapter 7, Section /15 POT Construction/Manipulation b ca ed (b,5) (c,2) (b,5) (c,2) (b,5)(d,1) (b,5)(c,2) (d,1) (b,5)(c,2) (e,  ) POT construction (start = a)    Find smallest  always on top; then swap and bubble down (d,1) (b,5)(c,2) (e,  )  (b,5)(c,2) (d,1)  (c,2) (b,5) (e,  ) (c,2) (b,5)(e,2) Adjust distances  POT doubly linked to adjacency list nodes  bubble up and down, as needed, when going through list of chosen node to adjust values swapbubble downadjust no bubbling needed (c,2) (b,5)(e,2)  …

Discussion #35 Chapter 7, Section /15 Dijkstra’s Algorithm vs. Floyd’s Dijkstra’s algorithm is O(mlogn) = O(n 2 logn) in the worst case. Floyd’s algorithm is O(n 3 ), but computes all shortest paths. Dijkstra’s algorithm can compute all shortest paths by starting it n times, once for each node. Thus, to compute all paths, Dijkstra’s algorithm is O(nmlogn) = O(n 3 logn) in the worst case. Which is better depends on the application. –Dijkstra’s algorithm is better if only a path from a single node to another or to all others is needed. –For all paths, Dijkstra’s algorithm is better for sparse graphs, and Floyd’s algorithm is better for dense graphs.