Shortest Paths Definitions Single Source Algorithms Bellman Ford DAG shortest path algorithm Dijkstra All Pairs Algorithms Using Single Source Algorithms Matrix multiplication Floyd-Warshall Both of above use adjacency matrix representation and dynamic programming Johnson’s algorithm Uses adjacency list representation

Single Source Definition Input Weighted, connected directed graph G=(V,E) Weight (length) function w on each edge e in E Source node s in V Task Compute a shortest path from s to all nodes in V

All Pairs Definition Input Task Weighted, connected directed graph G=(V,E) Weight (length) function w on each edge e in E We will typically assume w is represented as a matrix Task Compute a shortest path from all nodes in V to all nodes in V

Comments If edges are not weighted, then BFS works for single source problem Optimal substructure A shortest path between s and t contains other shortest paths within it No known algorithm is better at finding a shortest path from s to a specific destination node t in G than finding the shortest path from s to all nodes in V

Negative weight edges Negative weight edges can be allowed as long as there are no negative weight cycles If there are negative weight cycles, then there cannot be a shortest path from s to any node t (why?) If we disallow negative weight cycles, then there always is a shortest path that contains no cycles

Relaxation technique For each vertex v, we maintain an upper bound d[v] on the weight of shortest path from s to v d[v] initialized to infinity Relaxing an edge (u,v) Can we improve the shortest path to v by going through u? If d[v] > d[u] + w(u,v), d[v] = d[u] + w(u,v) This can be done in O(1) time

Bellman-Ford Algorithm 3 5 7 -5 2 1 s Bellman-Ford (G, w, s) Initialize-Single-Source(G,s) for (i=1 to V-1) for each edge (u,v) in E relax(u,v); if d[v] > d[u] + w(u,v) return NEGATIVE WEIGHT CYCLE 3 5 -1 -5 2 1 s

Running Time for (i=1 to V-1) The above takes (V-1)O(E) = O(VE) time for each edge (u,v) in E relax(u,v); The above takes (V-1)O(E) = O(VE) time if d[v] > d[u] + w(u,v) return NEGATIVE WEIGHT CYCLE The above takes O(E) time

Proof of Correctness If there is a shortest path from s to any node v, then d[v] will have this weight at end Let p = (e1, e2, …, ek) = (v1, v2, v3, …, v,+1) be a shortest path from s to v s = v1, v = vk+1, ei = (vi, vi+1) At beginning of ith iteration, during which we relax edge ei, d[vi] must be correct, so at end of ith iteration, d[vi+1] will e correct Iteration is the full sweep of relaxing all edges Proof by induction

Negative weight cycle for each edge (u,v) in E if d[v] > d[u] + w(u,v) return NEGATIVE WEIGHT CYCLE If no neg weight cycle, d[v] ≤ d[u] + w(u,v) for all (u,v) If there is a negative weight cycle C, for some edge (u,v) on C, it must be the case that d[v] > d[u] + w(u,v). Suppose this is not true for some neg. weight cycle C sum these (d[v] ≤ d[u] + w(u,v)) all the way around C We end up with sum(d[v]) ≤ sum(d[u]) + weight(C) This is impossible unless weight(C) = 0 But weight(C) is negative, so this cannot happen Thus for some (u,v) on C, d[v] > d[u] + w(u,v)

DAG shortest path algorithm 5 DAG-SP (G, w, s) Initialize-Single-Source(G,s) Topologically sort vertices in G for each vertex u, taken in sorted order for each edge (u,v) in E relax(u,v); 3 4 -4 s 1 5 3 8

Running time Improvement O(V+E) for the topological sorting We only do 1 relaxation for each edge: O(E) time for each vertex u, taken in sorted order for each edge (u,v) in E relax(u,v); Overall running time: O(V+E)

Proof of Correctness If there is a shortest path from s to any node v, then d[v] will have this weight at end Let p = (e1, e2, …, ek) = (v1, v2, v3, …, v,+1) be a shortest path from s to v s = v1, v = vk+1, ei = (vi, vi+1) Since we sort edges in topological order, we will process node vi (and edge ei) before processing later edges in the path.

Dijkstra’s Algorithm Dijkstra (G, w, s) /* Assumption: all edge weights non-negative */ Initialize-Single-Source(G,s) Completed = {}; ToBeCompleted = V; While ToBeCompleted is not empty u =EXTRACT-MIN(ToBeCompleted); Completed += {u}; for each edge (u,v) relax(u,v); 15 5 7 4 3 2 1 s

Running Time Analysis While ToBeCompleted is not empty u =EXTRACT-MIN(ToBeCompleted); Completed += {u}; for each edge (u,v) relax(u,v); Each edge relaxed at most once: O(E) Need to decrease-key potentially once per edge Need to extract-min once per node

Running Time Analysis cont’d O(E) edge relaxations Priority Queue operations O(E) decrease key operations O(V) extract-min operations Three implementations of priority queues Array: O(V2) time decrease-key is O(1) and extract-min is O(V) Binary heap: O(E log V) time assuming E >= V decrease-key and extract-min are O(log V) Fibonacci heap: O(V log V + E) time decrease-key is O(1) amortized time and extract-min is O(log V)

Compare Dijkstra’s algorithm to Prim’s algorithm for MST Dijsktra Priority Queue operations O(E) decrease key operations O(V) extract-min operations Prim O(E) decrease-key operations Is this a coincidence or is there something more here?

Proof of Correctness s u v Only nodes in S Assume that Dijkstra’s algorithm fails to compute length of all shortest paths from s Let v be the first node whose shortest path length is computed incorrectly Let S be the set of nodes whose shortest paths were computed correctly by Dijkstra prior to adding v to the processed set of nodes. Dijkstra’s algorithm has used the shortest path from s to v using only nodes in S when it added v to S. The shortest path to v must include one node not in S Let u be the first such node. s u v Only nodes in S

Proof of Correctness The length of the shortest path to u must be at least that of the length of the path computed to v. Why? The length of the path from u to v must be < 0. No path can have negative length since all edge weights are non-negative, and thus we have a contradiction. s u v Only nodes in S

Computing paths (not just distance) Maintain for each node v a predecessor node p(v) p(v) is initialized to be null Whenever an edge (u,v) is relaxed such that d(v) improves, then p(v) can be set to be u Paths can be generated from this data structure

All pairs algorithms using single source algorithms Call a single source algorithm from each vertex s in V O(V X) where X is the running time of the given algorithm Dijkstra linear array: O(V3) Dijkstra binary heap: O(VE log V) Dijkstra Fibonacci heap: O(V2 log V + VE) Bellman-Ford: O(V2 E) (negative weight edges)

Two adjacency matrix based algorithms Matrix-multiplication based algorithm Let Lm(i,j) denote the length of the shortest path from node i to node j using at most m edges What is our desired result in terms of Lm(i,j)? What is a recurrence relation for Lm(i,j)? Floyd-Warshall algorithm Let Lk(i,j) denote the length of the shortest path from node i to node j using only nodes within {1, …, k} as internal nodes. What is our desired result in terms of Lk(i,j)? What is a recurrence relation for Lk(i,j)?

Conceptual pictures i k j i k j OR Shortest path using at most 2m edges Try all possible nodes k i k j Shortest path using at most m edges Shortest path using at most m edges Shortest path using nodes 1 through k Shortest path using nodes 1 through k-1 i k j Shortest path using nodes 1 through k-1 OR Shortest path using nodes 1 through k-1

Running Times Matrix-multiplication based algorithm O(V3 log V) log V executions of “matrix-matrix” multiplication Not quite matrix-matrix multiplication but same running time Floyd-Warshall algorithm O(V3) V iterations of an O(V2) update loop The constant is very small, so this is a “fast” O(V3)

Johnson’s Algorithm Key ideas Reweight edge weights to eliminate negative weight edges AND preserve shortest paths Use Bellman-Ford and Dijkstra’s algorithms as subroutines Running time: O(V2 log V + VE) Better than earlier algorithms for sparse graphs

Reweighting Original edge weight is w(u,v) New edge weight: w’(u,v) = w(u,v) + h(u) – h(v) h(v) is a function mapping vertices to real numbers Key observation: Let p be any path from node u to node v w’(p) = w(p) + h(u) – h(v)

Computing vertex weights h(v) Create a new graph G’ = (V’, E’) by adding a new vertex s to V adding edges (s,v) for all v in V with w(s,v) = 0 Set h(v) to be the length of the shortest path from this new node s to node v This is well-defined if G’ does not contain negative weight cycles Note that h(v) ≤ h(u) + w(u,v) for all (u,v) in E’ Thus, w’(u,v) = w(u,v) + h(u) – h(v) ≥ 0 3 4 7 -5 -7 1

Algorithm implementation Run Bellman-Ford on G’ from new node s If no negative weight cycle, then use h(v) values from Bellman-Ford Now compute w’(u,v) for each edge (u,v) in E Now run Dijkstra’s algorithm using w’ Use each node as source node Modify d[u,v] at end by adding h(v) and subtracting h(u) to get true path weight Running time: O(VE) [from one run of Bellman-Ford] + O(V2 log V + VE) [from V runs of Dijkstra]