1. More Algorithm Examples Mani Chandy mani@cs.caltech.edu

2. Goal for Today Review data structures, dynamic programming and greedy algorithms by applying them to new problems. Creative step: decide which tools to apply for a given problem.

3. Single-Source Shortest Path 1 6 2 3 5 4 2 5 6 1 2 1 4 2 3 1 4 Find shortest path from vertex 1 to all vertices. All edge weights are non-negative

4. Single-Source vs All-Points 1.For single source the algorithm computes the shortest paths from a single specified vertex whereas in all-points the shortest paths between all pairs of vertices is computed. 2.For single source we are assuming that there are no negative weights whereas for all-points we allowed negative weights.

5. Data Structure Assumption To begin with, assume that the graph is specified by an edge weight matrix W, where W[j,k] is the weight of the edge from vertex j to vertex k. The memory required by the matrix is proportional to N 2 A more efficient (in terms of time and space) implementation for sparse graphs is to use an array of adjacency lists. AdjacenyList[j] is a list of neighboring vertices of vertex j. the amount of storage is proportional to E, the number of edges, and not to N 2 (where N is the number of vertices).

6. Creative Step What tool shall we apply? –Dynamic programming –Greedy algorithms –Clevel data structures Let’s postpone the data structure design issue. Sometimes greedy algorithms and dynamic programming are similar. Try greedy.

7. What do non-negative weights give? Crucial insight: There exists a shortest path to the k-th nearest vertex that has intermediate vertices only from the set of (k-1) nearest vertices. Proof: Consider a path from the source to the j-th nearest Vertex, let us call it u, and then on an edge from u to the k-th nearest vertex, let us call it v. for j > k. v u source

8. Proof of Crucial Insight v u source The length of this path is the length of the path to u + W[u,v] Since W[u,v] is non-negative, the length of this path is at least the length of the path to u.

9. The Greedy Decision Make a decision that is optimal, and from which we can make subsequent decisions. The j-th greedy decision: identify the vertex that is j-th closest to the source.

10. The Greedy Algorithm Intuition: have a weight, say 1Kg, at each node, and have strings between weights, where the length of the string is the weight of the corresponding edge. 1 3 2 source Some strings will be lose. Some strings will be taut. Closest vertexNext closestThird closest

11. The Greedy Steps 1 3 2 4 Identified closest At each greedy step, the next closest vertex is identified. progression of greedy steps

12. The Crucial Insight 1 3 2 4 To find the distance to the k-th nearest vertex we do not need to consider vertices farther away as intermediate vertices. 3-rd nearest vertex We don’t need to consider the 4-th nearest vertex as a possible intermediate vertex in computing distance to 3-rd nearest.

13. The Creative Step 1 3 2 Step j, we have identified the j closest vertices. What information do we need to identify the next closest vertex?

14. Data Structures Let L[t] be the set of the j closest vertices identified at step t. 1 3 2 4 L[0] = {} L[1] = {1} L[2] = {1,3} L[3] = {1,3,2} L[4] = {1,3,2,4}

15. Recursion Relation for L L[t+1] = L[t] union {v} –Where v is the (t+1)-th closest vertex. How do we find the (t+1)-th closest vertex?

16. More Data Structures Let D[j,t] be the shortest distance to vertex j over all paths for which the intermediate vertices (if any) are from set L[t]. A C B E D L[2] = {A,C} D[B,2]: Restrict attention to paths from source, vertex A, to vertex B that can only pass through vertex C, and do not pass through vertices B, E, D.

17. Theorem For all j in L[t]: D[j,t] is the shortest distance to j over all paths Recall D[j,t] is the shortest distance to vertex j over all paths for which the intermediate vertices (if any) are from set L[t]. This theorem tells us that for j in L[t], D[j,t] is the shortest distance to j regardless of the intermediate vertices. Proof follows directly from the crucial insight.

18. Recursion for D[j,t] Recall L[t+1] = L[t] union {v} where v is the (t+1)-th closest vertex. D[j,t+1] = min(D[j,t], D[v,t] + W[v,j]) Proof ?

19. Proof for Recursion of D[j,t+1] Partition the set of paths from the source to vertex j that can only pass through vertices in set L[t+1] into two subsets: the paths that can only pass through vertices in L[t] the paths that pass though v and can pass through vertices in L[t] The distance of paths in the first subset is at least D[j,t] from the induction hypothesis. We must prove that the distance of paths in the second subset is at least D[v,t]+W[v,j]

20. Proof for Recursion of D[j,t+1] We must prove that the distance of paths in the second subset is at least D[v,t]+W[v,j] Partition the paths that pass through v on their way to j into Two subsets: 1.Paths that go through v and then go to some other intermediate vertex in L[t] and then to j. 2. Paths that go to v and then directly to j. Consider a path in the first subset: from source to v to some vertices in L[t] with a vertex k being the prefinal vertex. There is an optimal path to k without going through v (from the induction hypothesis). So ignore the first subset.

21. The Recursive Relations L[t+1] = L[t] union {v} where v is ? v is a vertex that is not in L[t], and where D[v, t+1] is the minimum over all vertices not in L[t] of D[j,r+1] D[j, t+1] = min(D[j,t], D[v,t] + W[v,j]) For j not in L[t+1], D[j, t+1] is ?:

22. Initial Conditions L[0] = {}, the empty set. D[j,0] = 0 if j is the source, I (infinity) if j is not the source.

23. The Algorithm in Action 1 6 2 3 5 4 2 5 6 1 2 1 4 2 3 1 4 0 2 3 5 6 7 0 I I I I I 1 2 3 4 5 6 0 5 I I 6 2 0 2 0 3 6 I 6 23 0 3 6 7 5 25 6 7 Array D

24. Proof Assume induction hypothesis for t = 0 … r and prove the hypothesis for t= r+1 D[j,t] is the shortest distance from vertex 1 to vertex j that can have intermediate vertices only from set L[t]. The vertex set L[t] is the set of vertices at step t for which D[j,t] is the shortest distance from vertex 1 to vertex j. The vertices in L[t] are closer or as close to the source as the Vertices that are not in L[t]. The Induction Hypothesis

25. Proof of Induction Step Most of the steps have already been proved. Details are left to you.

26. The Homework Problem 1234 modules wires Given C[j,k] the number of wires between modules j and k, order the modules in a straight line to minimize wire length. Note C[j,k] = C[k,j]