1. CS 473 All Pairs Shortest Paths1 CS473 – Algorithms I All Pairs Shortest Paths

2. CS 473 All Pairs Shortest Paths2 All Pairs Shortest Paths (APSP) given : directed graph G = ( V, E ), weight function ω : E → R, |V| = n goal : create an n × n matrix D = ( d ij ) of shortest path distances i.e., d ij = δ ( v i, v j ) trivial solution : run a SSSP algorithm n times, one for each vertex as the source.

3. CS 473 All Pairs Shortest Paths3 All Pairs Shortest Paths (APSP) ► all edge weights are nonnegative : use Dijkstra’s algorithm –PQ = linear array : O ( V 3 + VE ) = O ( V 3 ) –PQ = binary heap : O ( V 2 lgV + EVlgV ) = O ( V 3 lgV ) for dense graphs better only for sparse graphs –PQ = fibonacci heap : O ( V 2 lgV + EV ) = O ( V 3 ) for dense graphs better only for sparse graphs ► negative edge weights : use Bellman-Ford algorithm –O ( V 2 E ) = O ( V 4 ) on dense graphs

4. CS 473 All Pairs Shortest Paths4 Adjacency Matrix Representation of Graphs ►n x n matrix W = (ω ij ) of edge weights : ω( v i, v j ) if ( v i, v j )  E ω ij = ∞ if ( v i, v j )  E ►assume ω ii = 0 for all v i  V, because –no neg-weight cycle  shortest path to itself has no edge, i.e., δ ( v i,v i ) = 0

5. CS 473 All Pairs Shortest Paths5 Dynamic Programming (1) Characterize the structure of an optimal solution. (2) Recursively define the value of an optimal solution. (3) Compute the value of an optimal solution in a bottom-up manner. (4) Construct an optimal solution from information constructed in (3).

6. CS 473 All Pairs Shortest Paths6 Shortest Paths and Matrix Multiplication Assumption : negative edge weights may be present, but no negative weight cycles. (1) Structure of a Shortest Path : Consider a shortest path p ij m from v i to v j such that |p ij m | ≤ m ► i.e., path p ij m has at most m edges. no negative-weight cycle  all shortest paths are simple  m is finite  m ≤ n – 1 i = j  |p ii | = 0 & ω( p ii ) = 0 i ≠ j  decompose path p ij m into p ik m-1 & v k → v j, where |p ik m-1 | ≤ m - 1 ► p ik m-1 should be a shortest path from v i to v k by optimal substructure property. ► Therefore, δ (v i,v j ) = δ (v i,v k ) + ω k j

7. CS 473 All Pairs Shortest Paths7 Shortest Paths and Matrix Multiplication (2) A Recursive Solution to All Pairs Shortest Paths Problem : d ij m = minimum weight of any path from v i to v j that contains at most “ m ” edges. m = 0 : There exist a shortest path from v i to v j with no edges ↔ i = j. 0 if i = j ► d ij 0 = ∞ if i ≠ j m ≥ 1 : d ij m = min { d ij m-1, min 1≤k≤n Λ k≠j { d ik m-1 + ω kj }} = min 1≤k≤n {d ik m-1 + ω kj } for all v k  V, since ω j j = 0 for all v j  V.

8. CS 473 All Pairs Shortest Paths8 Shortest Paths and Matrix Multiplication to consider all possible shortest paths with ≤ m edges from v i to v j ► consider shortest path with ≤ m -1 edges, from v i to v k, where v k  R v i and (v k,v j )  E note : δ (v i,v j ) = d ij n-1 = d ij n = d ij n+1, since m ≤ n -1 = | V | - 1 vivi vjvj v k ’s

9. CS 473 All Pairs Shortest Paths9 Shortest Paths and Matrix Multiplication (3) Computing the shortest-path weights bottom-up : given W = D 1, compute a series of matrices D 2, D 3,..., D n-1, where D m = ( d ij m ) for m = 1, 2,..., n-1 ► final matrix D n-1 contains actual shortest path weights, i.e., d ij n-1 = δ (v i,v j ) SLOW-APSP( W ) D 1 ← W for m ← 2 to n-1 do D m ← EXTEND( D m-1, W ) return D n-1

10. CS 473 All Pairs Shortest Paths10 Shortest Paths and Matrix Multiplication EXTEND ( D, W ) ► D = ( d ij ) is an n x n matrix for i ← 1 to n do for j ← 1 to n do d ij ← ∞ for k ← 1 to n do d ij ← min{d ij, d ik + ω k j } return D MATRIX-MULT ( A, B ) ► C = ( c ij ) is an n x n result matrix for i ←1 to n do for j ← 1 to n do c ij ← 0 for k ← 1 to n do c ij ← c ij + a ik x b k j return C

11. CS 473 All Pairs Shortest Paths11 Shortest Paths and Matrix Multiplication relation to matrix multiplication C = A B : c ij = ∑ 1≤k≤n a ik x b k j, ► D m-1 ↔ A & W ↔ B & D m ↔ C “min” ↔ “t” & “t” ↔ “x” & “∞” ↔ “0” Thus, we compute the sequence of matrix products D 1 = D 0 x W = W ; note D 0 = identity matrix, 0 if i = j D 2 = D 1 x W = W 2 i.e., d ij 0 = D 3 = D 2 x W = W 3 ∞ if i ≠ j D n-1 = D n-2 x W = W n-1 running time :  ( n 4 ) =  ( V 4 ) ► each matrix product :  ( n 3 ) ► number of matrix products : n-1

12. CS 473 All Pairs Shortest Paths12 Shortest Paths and Matrix Multiplication 1 2 3 45 2 1 4 3 8 -5 -4 6 7 Example

13. CS 473 All Pairs Shortest Paths13 Shortest Paths and Matrix Multiplication 12345 1038∞-4 2∞0∞17 3∞40∞∞ 42∞-50∞ 5∞∞∞60 D 1 = D 0 W 1 2 3 45 2 1 4 3 8 -5 -4 6 7

14. CS 473 All Pairs Shortest Paths14 Shortest Paths and Matrix Multiplication 12345 10382-4 230 17 3∞40511 42-50-2 58∞160 D 2 = D 1 W 1 2 3 45 2 1 4 3 8 -5 -4 6 7

15. CS 473 All Pairs Shortest Paths15 Shortest Paths and Matrix Multiplication 12345 103-32-4 230 1 3740511 42-50-2 585160 D 3 = D 2 W 1 2 3 45 2 1 4 3 8 -5 -4 6 7

16. CS 473 All Pairs Shortest Paths16 Shortest Paths and Matrix Multiplication 12345 101-32-4 230 1 374053 42 -50-2 585160 D 4 = D 3 W 1 2 3 45 2 1 4 3 8 -5 -4 6 7

17. CS 473 All Pairs Shortest Paths17 SSSP and Matrix-Vector Multiplication relation of APSP to one step of matrix multiplication cjcj cjcj riri d ij m = riri k k D m ↔ C D m-1 ↔ A W ↔ B

18. CS 473 All Pairs Shortest Paths18 SSSP and Matrix-Vector Multiplication d ij n-1 at row r i and column c j of product matrix = δ ( v i =s, v j ) for j = 1, 2, 3,..., n row r i of the product matrix = solution to single-source shortest path problem for s = v i. ► r i of C = matrix B multiplied by r i of A  D i m = D i m-1 x W

19. CS 473 All Pairs Shortest Paths19 SSSP and Matrix-Vector Multiplication 0 if i = j let D i 0 = d 0, where d j 0 = ∞ otherwise we compute a sequence of n-1 “matrix-vector” products d i 1 = d i 0 x W d i 2 = d i 1 x W d i 3 = d i 2 x W : d i n-1 = d i n-2 x W

20. CS 473 All Pairs Shortest Paths20 SSSP and Matrix-Vector Multiplication this sequence of matrix-vector products ► same as Bellman-Ford algorithm. ► vector d i m  d values of Bellman-Ford algorithm after m-th relaxation pass. ► d i m ← d i m-1 x W  m-th relaxation pass over all edges.

21. CS 473 All Pairs Shortest Paths21 SSSP and Matrix-Vector Multiplication BELLMAN-FORD ( G, v i ) ► perform RELAX ( u, v ) for ► every edge ( u, v )  E for j ← 1 to n do for k ← 1 to n do RELAX ( v k, v j ) RELAX ( u, v ) d v = min { d v, d u + ω uv } EXTEND ( d i, W ) ► d i is an n-vector for j ← 1 to n do d j ← ∞ for k ← 1 to n do d j ← min { d j, d k + ω kj }

22. CS 473 All Pairs Shortest Paths22 idea : goal is not to compute all D m matrices ► we are interested only in matrix D n-1 recall : no negative-weight cycles  D m = D n-1 for all m ≥ n-1 we can compute D n-1 with only lg(n-1) matrix products as D 1 = W D 2 = W 2 = W x W D 4 = W 4 = W 2 x W 2 D 8 = W 8 = W 4 x W 4 = = This technique is called repeated squaring. Improving Running Time Through Repeated Squaring

23. CS 473 All Pairs Shortest Paths23 Improving Running Time Through Repeated Squaring FASTER-APSP ( W ) D 1 ← W m ← 1 while m < n-1 do D 2m ← EXTEND ( D m, D m ) m ← 2m return D m final iteration computes D 2m for some n-1 ≤ 2m ≤ 2n-2  D 2m = D n-1 running time :  ( n 3 lgn ) =  ( V 3 lgV ) ► each matrix product :  ( n 3 ) ► # of matrix products : lg( n-1 ) ► simple code, no complex data structures, small hidden constants in  -notation.

24. CS 473 All Pairs Shortest Paths24 Idea Behind Repeated Squaring decompose p ij 2m as p ik m & p kj m, where p ij 2m : v i v j p ik m : v i v k p kj m : v k v j vivi vjvj vk’svk’s

25. CS 473 All Pairs Shortest Paths25 Floyd-Warshall Algorithm assumption : negative-weight edges, but no negative-weight cycles (1) The Structure of a Shortest Path : Definition : intermediate vertex of a path p = ► any vertex of p other than v 1 or v k. p ij m : a shortest path from v i to v j with all intermediate vertices from V m = { v 1, v 2,..., v m } relationship between p ij m and p ij m-1 ► depends on whether v m is an intermediate vertex of p ij m - case 1: v m is not an intermediate vertex of p ij m  all intermediate vertices of p ij m are in V m -1  p ij m = p ij m-1

26. CS 473 All Pairs Shortest Paths26 Floyd-Warshall Algorithm - case 2 : v m is an intermediate vertex of p ij m - decompose path as v i v m v j  p 1 : v i v m & p 2 : v m v j - by opt. structure property both p 1 & p 2 are shortest paths. - v m is not an intermediate vertex of p 1 & p 2  p 1 = p im m-1 & p 2 = p mj m-1 vivi vjvj vmvm p1p1 p2p2 VmVm

27. CS 473 All Pairs Shortest Paths27 Floyd-Warshall Algorithm (2) A Recursive Solution to APSP Problem : d ij m = ω(p ij ) : weight of a shortest path from v i to v j with all intermediate vertices from V m = { v 1, v 2,..., v m }. note : d ij n = δ (v i,v j ) since V n = V ► i.e., all vertices are considered for being intermediate vertices of p ij n.

28. CS 473 All Pairs Shortest Paths28 Floyd-Warshall Algorithm compute d ij m in terms of d ij k with smaller k < m m = 0 : V 0 = empty set  path from v i to v j with no intermediate vertex. i.e., v i to v j paths with at most one edge  d ij 0 = ω i j m ≥ 1 : d ij m = min {d ij m-1, d im m-1 + d mj m-1 }

29. CS 473 All Pairs Shortest Paths29 Floyd-Warshall Algorithm (3) Computing Shortest Path Weights Bottom Up : FLOYD-WARSHALL( W ) ►D 0, D 1,..., D n are n x n matrices for m ← 1 to n do for i ← 1 to n do for j ← 1 to n do d ij m ← min {d ij m-1, d im m-1 + d mj m-1 } return D n

30. CS 473 All Pairs Shortest Paths30 Floyd-Warshall Algorithm FLOYD-WARSHALL ( W ) ► D is an n x n matrix D ← W for m ← 1 to n do for i ← 1 to n do for j ← 1 to n do if d ij > d im + d mj then d ij ← d im + d mj return D

31. CS 473 All Pairs Shortest Paths31 Floyd-Warshall Algorithm maintaining n D matrices can be avoided by dropping all superscripts. – m-th iteration of outermost for-loop begins with D = D m-1 ends with D = D m –computation of d ij m depends on d im m-1 and d mj m-1. no problem if d im & d mj are already updated to d im m & d mj m since d im m = d im m-1 & d mj m = d mj m-1. running time :  ( n 3 ) =  ( V 3 ) simple code, no complex data structures, small hidden constants

32. CS 473 All Pairs Shortest Paths32 Transitive Closure of a Directed Graph G ' = ( V, E ' ) : transitive closure of G = ( V, E ), where ► E ' = { (v i, v j ): there exists a path from v i to v j in G } trivial solution : assign W such that 1 if (v i, v j )  E ω ij = ∞ otherwise ► run Floyd-Warshall algorithm on W ► d ij n < n  there exists a path from v i to v j, i.e., (v i, v j )  E ' ► d ij n = ∞  no path from v i to v i, i.e., (v i, v j )  E ' ► running time :  ( n 3 ) =  ( V 3 )

33. CS 473 All Pairs Shortest Paths33 Transitive Closure of a Directed Graph Better  ( V 3 ) algorithm : saves time and space. 1 if i = j or (v i, v j )  E ► W = adjacency matrix : ω ij = 0 otherwise ► run Floyd-Warshall algorithm by replacing “min” → “ ” & “+” → “ ” 1 if a path from v i to v j with all intermediate vertices from V m define t ij m = 0 otherwise ► t ij n = 1  (v i, v j )  E ' & t ij n = 0  (v i, v j )  E ' recursive definition for t ij m = t ij m-1 (t im m-1 t mj m-1 ) with t ij 0 = ω ij

34. CS 473 All Pairs Shortest Paths34 Transitive Closure of a Directed Graph T-CLOSURE (G ) ► T = ( t ij ) is an n x n boolean matrix for i ← 1 to n do for j ← 1 to n do if i = j or ( v i, v j )  E then t ij ← 1 else t ij ← 0 for m ← 1 to n do for i ← 1 to n do for j ← 1 to n do t ij ← t ij ( t im t mj )

35. CS 473 All Pairs Shortest Paths35 Johnson’s Algorithm for Sparse Graphs (1) Preserving shortest paths by edge reweighting : L1 : given G = ( V, E ) with ω : E → R ► let h : V → R be any weighting function on the vertex set ► define ω( ω, h ) : E → R as ω( u, v ) = ω( u, v ) + h (u) – h (v) ► let p 0k = be a path from v 0 to v k (a) ω( p 0k ) = ω( p 0k ) + h (v 0 ) - h (v k ) (b) ω( p 0k ) = δ(v 0, v k ) in ( G, ω )  ω( p 0k ) = δ(v 0, v k ) in ( G, ω ) (c) ( G, ω ) has a neg-wgt cycle  ( G, ω ) has a neg-wgt cycle

36. CS 473 All Pairs Shortest Paths36 Johnson’s Algorithm for Sparse Graphs proof (a): ω( p 0k ) = ∑ 1 ≤i ≤k ω( v i-1,v i ) = ∑ 1 ≤i ≤k ( ω(v i-1,v i ) + h (v 0 ) - h (v k ) ) = ∑ 1 ≤i ≤k ω(v i-1,v i ) + ∑ 1 ≤i ≤k ( h (v 0 ) - h (v k ) ) = ω( p 0k ) + h (v 0 ) - h (v k ) proof (b): (  ) show ω( p 0k ) = δ ( v 0, v k )  ω( p 0k ) = δ ( v 0, v k ) by contradiction. ► Suppose that a shorter path p 0k ' from v 0 to v k in (G, ω ), then ω( p 0k ' ) < ω( p 0k ) due to (a) we have –ω( p 0k ' ) + h (v 0 ) - h (v k ) = ω( p 0k ' ) < ω( p 0k ) = ω( p 0k ) + h (v 0 ) - h (v k ) ω( p 0k ' ) + h (v 0 ) - h (v k ) < ω( p 0k ) + h (v 0 ) - h (v k ) ω( p 0k ' ) < ω( p 0k )  contradicts that p 0k is a shortest path in ( G, ω )

37. CS 473 All Pairs Shortest Paths37 Johnson’s Algorithm for Sparse Graphs proof (b): (<=) similar proof (c): (  ) consider a cycle c =. Due to (a) ► ω(c ) = ∑ 1 ≤i ≤k ω(v i-1,v i ) = ω(c ) + h (v 0 ) - h (v k ) = ω(c ) + h (v 0 ) - h (v 0 ) = ω(c ) since v k = v 0 ► ω(c ) = ω(c ). QED

38. CS 473 All Pairs Shortest Paths38 Johnson’s Algorithm for Sparse Graphs (2) Producing nonnegative edge weights by reweighting : given (G, ω) with G = ( V, E ) and ω : E → R construct a new graph ( G ', ω ' ) with G ' = ( V ', E ' ) and ω ' = E ' → R ► V ' = V U { s } for some new vertex s  V ► E ' = E U { ( s,v ) : v  V } ► ω ' (u,v) = ω(u,v) (u,v)  E and ω ' (s,v) = 0, v  V vertex s has no incoming edges  s  R v for any v in V ► no shortest paths from u ≠ s to v in G ' contains vertex s ► ( G ', ω ' ) has no neg-wgt cycle  (G, ω) has no neg-wgt cycle

39. CS 473 All Pairs Shortest Paths39 Johnson’s Algorithm for Sparse Graphs suppose that G and G ' have no neg-wgt cycle L2 : if we define h (v) = δ (s,v ) v  V in G ' and ω according to L1. ► we will have ω(u,v) = ω(u,v) + h(u) – h(v) ≥ 0 v  V proof : for every edge (u, v)  E δ (s,v ) ≤ δ (s, u) + ω(u, v) in G ' due to triangle inequality h (v) ≤ h (u) + ω(u, v)  0 ≤ ω(u, v) + h(u) – h(v) = ω(u, v)

40. CS 473 All Pairs Shortest Paths40 Johnson’s Algorithm for Sparse Graphs example : 2 1 4 3 8 -5 -4 6 7 0 0 0 0 0 v1v1 v2v2 v3v3 v4v4 v5v5 v0 = sv0 = s  ( G ', ω ' )

41. CS 473 All Pairs Shortest Paths41 Johnson’s Algorithm for Sparse Graphs Edge Reweighting 0 -5 0 -4 2 1 4 3 8 -5 -4 6 7 v1v1 v2v2 v3v3 v4v4 v5v5  (G ', ω ' ) with h(v)

42. CS 473 All Pairs Shortest Paths42 Johnson’s Algorithm for Sparse Graphs Edge Reweighting  (G, ω ) 2 0 0 4 13 0 0 2 10 v1v1 v2v2 v3v3 v4v4 v5v5

43. CS 473 All Pairs Shortest Paths43 Johnson’s Algorithm for Sparse Graphs Computing All-Pairs Shortest Paths adjacency list representation of G. returns n x n matrix D = ( d ij ) where d ij = δ ij, or reports the existence of a neg-wgt cycle.

44. CS 473 All Pairs Shortest Paths44 Johnson’s Algorithm for Sparse Graphs JOHNSON(G,ω) ► D=(d ij ) is an nxn matrix ► construct ( G ' = (V ', E ' ), ω ' ) s.t. V ' = V U {s}; E ' = E U { (s,v) : v  V } ► ω ' (u,v) = ω(u,v), (u,v)  E & ω ' (s,v) = 0 v  V if BELLMAN-FORD(G ', ω ', s) = FALSE then return “negative-weight cycle” else for each vertex v  V ' - {s} = V do h[v] ← d ' [v] ► d ' [v] = δ ' (s,v) computed by BELLMAN-FORD(G ', ω ', s) for each edge (u,v)  E do ω(u,v) ← ω(u,v) + h[u] – h[v] ► edge reweighting for each vertex u  V do run DIJKSTRA(G, ω, u) to compute d[v] = δ (u,v) for all v in V  (G,ω) for each vertex v  V do d uv = d[v] – ( h[u] – h[v] ) return D

45. CS 473 All Pairs Shortest Paths45 Johnson’s Algorithm for Sparse Graphs running time : O ( V 2 lgV + EV ) ► edge reweighting BELLMAN-FORD(G ', ω ', s) : O ( EV ) computing ω values : O (E ) ► |V| runs of DIJKSTRA : | V | x O ( VlgV + EV ) = O ( V 2 lgV + EV ); PQ = fibonacci heap