1. 1 Dynamic Programming Jose Rolim University of Geneva

2. Dynamic ProgrammingJose Rolim2 All pair shortest path  Use Bellman-Ford V times  O ( E )  Non-negative weights:  Use Dijkstra’s alg. V times O( log V + VE)  Can we do better for general graphs  i.e., weighted directed graph with possible negative weights

3. Dynamic ProgrammingJose Rolim3 Terminologies  G = (V, E) where V are numbered 1, 2,.. n=|V|  Matrix L with L(i,j)= the length of edge (i,j) and  0 if i=j   if the edge does not exist  L(i,j)  0 for all i,j  Output matrix W(i,j) with the distances

4. Dynamic ProgrammingJose Rolim4 Dynamic programming approach  Optimal substructure property holds:  If k is a node on the shortest path from i to j then the paths: from i to k must be optimal from k to j must be optimal

5. Dynamic ProgrammingJose Rolim5 Recursive solution  Let D be a matrix with the length of the shortest paths  Initially D=L  After k interactions D gives the length of the shortest paths that uses only nodes from 1 to k  After n interactions D gives the solution

6. Dynamic ProgrammingJose Rolim6 Recursive solution  At the kth interaction there are two options for the path between i and j:  the path does not pass by the k node and then: d(i,j) does not change between the two inteactions otherwise d(i,j)= d(i,k)+d(k,j)

7. Dynamic ProgrammingJose Rolim7 Recursive solution  Therefore  Dk[i,j]=min(Dk-1[i,j], Dk-1[i,k]+Dk-1[k,j])  and the algorithm finishes when k=n  ordering ???

8. Dynamic ProgrammingJose Rolim8 Floyd’s algorithm  function Floyd(L[1..n,1..n])  D=L  for k=1 to n do for i=1 to n do  for j=1 to n do D[i,j]= min (D[i,j], D[i,k]+D[k,j])  return D

9. Dynamic ProgrammingJose Rolim9 Remarks  Complexity O(n**3)  To get the paths we can use a second matrix for the nodes k used in the path  Negative lenghts?  Longest paths?

10. Dynamic ProgrammingJose Rolim10 Review: Dynamic programming  DP is a method for solving certain kind of problems  DP can be applied when the solution of a problem includes solutions to subproblems  We need to find a recursive formula for the solution  We can recursively solve subproblems, starting from the trivial case, and save their solutions in memory in a convenient order  In the end we’ll get the solution of the whole problem

11. Dynamic ProgrammingJose Rolim11 Properties of a problem that can be solved with dynamic programming  Simple Subproblems  We should be able to break the original problem to smaller subproblems that have the same structure  Optimal Substructure of the problems  The solution to the problem must be a composition of subproblem solutions  Subproblem Overlap  Optimal subproblems to unrelated problems can contain subproblems in common

12. Dynamic ProgrammingJose Rolim12 0-1 Knapsack problem  Given a knapsack with maximum capacity W, and a set S consisting of n items  Each item i has some weight w i and benefit value b i (all w i, b i and W are integer values)  Problem: How to pack the knapsack to achieve maximum total value of packed items?

13. Dynamic ProgrammingJose Rolim13 0-1 Knapsack problem: a picture W = 20 wiwi bibi 10 9 8 5 5 4 4 3 3 2 WeightBenefit value This is a knapsack Max weight: W = 20 Items

14. Dynamic ProgrammingJose Rolim14 0-1 Knapsack problem  Problem, in other words, is to find  The problem is called a “0-1” problem, because each item must be entirely accepted or rejected.  Just another version of this problem is the “Fractional Knapsack Problem”, where we can take fractions of items.

15. Dynamic ProgrammingJose Rolim15 Brute-force approach Let’s first solve this problem with a straightforward algorithm  Since there are n items, there are 2 n possible combinations of items.  We go through all combinations and find the one with the most total value and with total weight less or equal to W  Running time will be O(2 n )

16. Dynamic ProgrammingJose Rolim16 Brute-force approach  Can we do better?  Yes, with an algorithm based on dynamic programming  We need to carefully identify the subproblems Let’s try this: If items are labeled 1..n, then a subproblem would be to find an optimal solution for S k = {items labeled 1, 2,.. k}

17. Dynamic ProgrammingJose Rolim17 Defining a Subproblem  This is a valid subproblem definition.  The question is: can we describe the final solution (S n ) in terms of subproblems (S k )?  Unfortunately, we can’t do that.

18. Dynamic ProgrammingJose Rolim18 Defining a Subproblem Max weight: W = 20 For S 4 : Total weight: 14; total benefit: 20 w 1 =2 b 1 =3 w 2 =4 b 2 =5 w 3 =5 b 3 =8 w 4 =3 b 4 =4 wiwi bibi 10 85 54 43 32 9 Item # 4 3 2 1 5 S4S4 S5S5 w 1 =2 b 1 =3 w 2 =4 b 2 =5 w 3 =5 b 3 =8 w 4 =9 b 4 =10 For S 5 : Total weight: 20 total benefit: 26 Solution for S 4 is not part of the solution for S 5 !!!

19. Dynamic ProgrammingJose Rolim19 Defining a Subproblem  As we have seen, the solution for S 4 is not part of the solution for S 5  So our definition of a subproblem is flawed and we need another one!  Let’s add another parameter: w, which will represent the exact weight for each subset of items  The subproblem then will be to compute B[k,w]

20. Dynamic ProgrammingJose Rolim20 Recursive Formula for subproblems  It means, that the best subset of S k that has total weight w is one of the two: 1) the best subset of S k-1 that has total weight w, or 2) the best subset of S k-1 that has total weight w-w k plus the item k  Recursive formula for subproblems:

21. Dynamic ProgrammingJose Rolim21 Recursive Formula  The best subset of S k that has the total weight w, either contains item k or not.  First case: w k >w. Item k can’t be part of the solution, since if it was, the total weight would be > w, which is unacceptable  Second case: w k <=w. Then the item k can be in the solution, and we choose the case with greater value

22. Dynamic ProgrammingJose Rolim22 0-1 Knapsack Algorithm for w = 0 to W B[0,w] = 0 for i = 0 to n B[i,0] = 0 for w = 0 to W if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w

23. Dynamic ProgrammingJose Rolim23 Running time for w = 0 to W B[0,w] = 0 for i = 0 to n B[i,0] = 0 for w = 0 to W What is the running time of this algorithm? O(W) Repeat n times O(n*W) Remember that the brute-force algorithm takes O(2 n )

24. Dynamic ProgrammingJose Rolim24 Example Let’s run our algorithm on the following data: n = 4 (# of elements) W = 5 (max weight) Elements (weight, benefit): (2,3), (3,4), (4,5), (5,6)

25. Dynamic ProgrammingJose Rolim25 Example (2) for w = 0 to W B[0,w] = 0 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 4

26. Dynamic ProgrammingJose Rolim26 Example (3) for i = 0 to n B[i,0] = 0 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 4

27. Dynamic ProgrammingJose Rolim27 Example (4) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=1 b i =3 w i =2 w=1 w-w i =-1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0

28. Dynamic ProgrammingJose Rolim28 Example (5) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=1 b i =3 w i =2 w=2 w-w i =0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3

29. Dynamic ProgrammingJose Rolim29 Example (6) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=1 b i =3 w i =2 w=3 w-w i =1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3

30. Dynamic ProgrammingJose Rolim30 Example (7) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=1 b i =3 w i =2 w=4 w-w i =2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3 3

31. Dynamic ProgrammingJose Rolim31 Example (8) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=1 b i =3 w i =2 w=5 w-w i =2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3 3 3

32. Dynamic ProgrammingJose Rolim32 Example (9) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=2 b i =4 w i =3 w=1 w-w i =-2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3 3 3 0

33. Dynamic ProgrammingJose Rolim33 Example (10) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=2 b i =4 w i =3 w=2 w-w i =-1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3 3 3 0 3

34. Dynamic ProgrammingJose Rolim34 Example (11) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=2 b i =4 w i =3 w=3 w-w i =0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3 3 3 0 3 4

35. Dynamic ProgrammingJose Rolim35 Example (12) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=2 b i =4 w i =3 w=4 w-w i =1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3 3 3 0 3 4 4

36. Dynamic ProgrammingJose Rolim36 Example (13) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=2 b i =4 w i =3 w=5 w-w i =2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3 3 3 0 3 4 4 7

37. Dynamic ProgrammingJose Rolim37 Example (14) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=3 b i =5 w i =4 w=1..3 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 3 3 3 3 00 3 4 4 7 0 3 4

38. Dynamic ProgrammingJose Rolim38 Example (15) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=3 b i =5 w i =4 w=4 w- w i =0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 000 3 4 4 7 0 3 4 5 3 3 3 3

39. Dynamic ProgrammingJose Rolim39 Example (15) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=3 b i =5 w i =4 w=5 w- w i =1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 000 3 4 4 7 0 3 4 5 7 3 3 3 3

40. Dynamic ProgrammingJose Rolim40 Example (16) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=3 b i =5 w i =4 w=1..4 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 000 3 4 4 7 0 3 4 5 7 0 3 4 5 3 3 3 3

41. Dynamic ProgrammingJose Rolim41 Example (17) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w 0 0 0 0 0 0 W 0 1 2 3 4 5 i 0123 0000 i=3 b i =5 w i =4 w=5 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 000 3 4 4 7 0 3 4 5 7 0 3 4 5 7 3 3 3 3

42. Dynamic ProgrammingJose Rolim42 Comments  This algorithm only finds the max possible value that can be carried in the knapsack  To know the items that make this maximum value, an addition to this algorithm is necessary

43. Dynamic ProgrammingJose Rolim43 Pseudo-polynomial algorithm  Running time (Dynamic Programming algorithm vs. naïve algorithm):  0-1 Knapsack problem: O(W*n) vs. O(2 n )  It is not polynomial algorithm, because: W = O(2 n )

44. Dynamic ProgrammingJose Rolim44 Longest common subsequence  Longest common subsequence (LCS) problem:  Given two sequences x[1..m] and y[1..n], find the longest subsequence which occurs in both  Ex: x = {A B C B D A B }, y = {B D C A B A}  {B C} and {A A} are both subsequences of both What is the LCS?

45. Dynamic ProgrammingJose Rolim45 Brute force algorithm  Brute-force algorithm:  For every subsequence of x, check if it’s a subsequence of y How many subsequences of x are there? What will be the running time of the brute-force algorithm?

46. Dynamic ProgrammingJose Rolim46 Running time: brute force algorithm  Brute-force algorithm:  2 m subsequences of x to check against n elements of y:  O(n 2 m )

47. Dynamic ProgrammingJose Rolim47 LCS Algorithm  Let’s only worry about the problem of finding the length of LCS  Define c[i,j] to be the length of the LCS of x[1..i] and y[1..j]  Initial cases: c[0,j] = c[i,0] = 0

48. Dynamic ProgrammingJose Rolim48 LCS Algorithm  Recursive formula for LCS:  c[i,j]=  c[i-1,j-1] + 1 if x[i]=y[j]  max{ c[i,j-1], c[i-1,j] } otherwise  Final answer c[m,n]

49. Dynamic ProgrammingJose Rolim49 Running time  After we have filled the array c[ ], we can backtrack to find the characters that constitute the Longest Common Subsequence  Algorithm runs in O(m*n), which is much better than the brute-force algorithm: O(n 2 m )

50. Dynamic ProgrammingJose Rolim50 Conclusions  Dynamic Programming is an algorithm design method that can be used when the solution to a problem may be viewed as the result of a sequence of decisions (Ordering) and:

51. Dynamic ProgrammingJose Rolim51 Conclusions  Principle of optimality: Suppose that in solving a problem, we have to make a sequence of decisions D 1, D 2, …, D n. If this sequence is optimal, then the last k decisions, 1  k  n must be optimal.  In summary, if a problem can be described by this principle, then it can be solved by dynamic programming.