1. Asst. Prof. Dr. İlker Kocabaş
Dynamic Programming
Lecture 9
Asst. Prof. Dr. İlker Kocabaş

2. Dynamic Programming History
Bellman. Pioneered the systematic study of dynamic programming in the 1950s.
Etymology.
Dynamic programming = planning over time.
Secretary of Defense was hostile to mathematical research.
Bellman sought an impressive name to avoid confrontation.
"it's impossible to use dynamic in a pejorative sense"
"something not even a Congressman could object to"
Reference: Bellman, R. E. Eye of the Hurricane, An Autobiography.

3. Introduction
Dynamic Programming(DP) applies to optimiza- tion problems in which a set of choices must be made in order to arrive at an optimal solution.
As choices are made, subproblems of the same form arise.
DP is effective when a given problem may arise from more than one partial set of choices.
The key technique is to store the solution to each subproblem in case it should appear

4. Introduction (cont.)
Divide and Conquer algorithms partition the problem into independent subproblems.
Dynamic Programming is applicable when the subproblems are not independent.(In this case DP algorithm does more work than necessary)
Dynamic Programming algorithm solves every subproblem just once and then saves its answer in a table.

5. Dynamic Programming Applications
Areas.
Bioinformatics.
Control theory.
Information theory.
Operations research.
Computer science: theory, graphics, AI, systems, ….
Some famous dynamic programming algorithms.
Viterbi for hidden Markov models.
Unix diff for comparing two files.
Smith-Waterman for sequence alignment.
Bellman-Ford for shortest path routing in networks.
Cocke-Kasami-Younger for parsing context free grammars.

6. Assembly-line Scheduling

7. Assembly-line Scheduling
If we are given a list of which stations to use in line 1 and which to use in line 2, then T(n)=Θ(n).
There are 2n possible ways to choose stations. Thus, the solution is obtained by enumarating all posible ways and computing how long each takes would take Ω(2n).

8. The steps of a dynamic programming
Characterize the structure of an optimal solution
Recursively define the value of an optimal solution
Compute the value of an optimal solution in a bottom-up fashion
Construct an optimal solution from computed information

9. Step 1:The structure of the fastest way through the factory
Consider the fastest possible way(fpw) to get from the starting point through station S1,j.
If j=1, there is only one way that the chassis could have gone.
For j=2, 3, ..., n there are two choices:
The chassis could have come from S1, j-1 and then directly to station S1, j.
The chassis could have come from S2, j-1 and then directly to station S1, j.

10. Step 1:The structure of the fastest way through the factory (cont.)
First, suppose that the fastest way(fw) through station S1, j is through station S1, j-1.
The key observation is that the chassis must have taken a fw from the starting point through station S1, j-1.

11. Step 1:The structure of the fastest way through the factory (cont.)
Similarly, suppose that the fw through station S2, j is through station S2, j-1.
We observe that the chassis must have taken a fw from tha starting point through station S2, j-1.

12. Step 1:The structure of the fastest way through the factory (cont.)
We can say that for assembly-line scheduling, an optimal solution to a problem contains within it an optimal solution to sub-problems (finding the fw through either S1, j-1 or S2, j-1).
We refer to this property as optimal substructure.

13. The fw through stations S1, j
The fw through station S1, j-1 and then directly through station S1, j. or
The fw through station S2, j-1 and then through station S1, j.

14. The fw through station S2, j
The fw through station S2, j-1 and then directly through station S2, j. or
The fw through station S1, j-1 and then through station S2, j.

15. Step 1:The structure of the fastest way through the factory (cont.)
To solve the problem of finding the fw through station j of either line, we solve the subproblems of finding the fw through station j-1 on both lines.
Thus, we can build an optimal solution to an instance of the assemly line scheduling problem by building on optimal solutions to subproblems

16. Step 2: A recursive solution
We define the value of an optimal solution recursively in tems of the optimal solutions to subproblems.
Let fi[j] denote the fastest possible time to get a chassis from the starting point throuhg station Si, j.
Let fi* denote the fastest time to get a chassis all the way through the factory.

17. Step 2: A recursive solution (cont.)

18. Step 2: A recursive solution (cont.)

19. Step 2: A recursive solution: An example
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 7 9 3 4 8 4 2 3 2 3 1 3 4
Chassis Enters 2 1 5 2 1 4 2 8 5 6 4
Line 2 5 7
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6 j j
f1[j]
l1[j]
f*=0
l* =0
f2[j]
l2[j]

20. Step 2: A recursive solution: An example
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 9 9 3 4 8 4 3 2 3 1 3 4
Chassis Enters 2 1 5 2 1 2 12 5 6 4
Line 2 5 7
Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j j
f1[j] 9
l1[j]
f*=0
l* =0
f2[j] 12
l2[j]

21. Step 2: A recursive solution: An example
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 9 18 3 4 8 4 3 3 1 3 4
Chassis Enters 1 5 2 1 2
Line 2 12 16 6 4 5 7
Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j j
f1[j] 9 18
l1[j] 1
f*=0
l* =0
f2[j] 12 16
l2[j] 1

22. Step 2: A recursive solution: An example
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 9 18 20 4 8 4 3 1 3 4
Chassis Enters 5 2 1 2
Line 2 12 16 22 4 5 7
Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j j
f1[j] 9 18 20
l1[j] 1 2
f*=0
l* =0
f2[j] 12 16 22
l2[j] 1 2

23. Step 2: A recursive solution: An example
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 9 18 20 24 8 4 3 3 4
Chassis Enters 2 1 2
Line 2 12 16 22 25 5 7
Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j j
f1[j] 9 18 20 24
l1[j] 1 2 1
f*=0
l* =0
f2[j] 12 16 22 25
l2[j] 1 2 1

24. Step 2: A recursive solution: An example
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 9 18 20 24 32 4 3 4
Chassis Enters 1 2
Line 2 12 16 22 25 30 7
Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j j
f1[j] 9 18 20 24 32
l1[j] 1 2 1 1
f*=0
l* =0
f2[j] 12 16 22 25 30
l2[j] 1 2 1 2

25. Step 2: A recursive solution: An example ( Correction !!! )
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 9 18 20 24 32 4 3 4
Chassis Enters 1 2
Line 2 12 16 22 25 30 7
Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j j
f1[j] 9 18 20 24 32
l1[j] 1 2 1 1
f*=0
l* =0
f2[j] 12 16 22 25 30
l2[j] 1 2 1 2

26. Step 2: A recursive solution: An example
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 9 18 20 20 24 32 35 3
Chassis Enters 2
Line 2 12 16 22 22 25 30 37
Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j j
f1[j] 9 18 20 24 32 35
l1[j] 1 2 1 1 1
f*=0
l* =0
f2[j] 12 16 22 25 30 37
l2[j] 1 2 1 2 2

27. Step 2: A recursive solution: An example
Station S1,1 Station S1,2 Station S1,3 Station S1,4 Station S1,5 Station S1,6
Line 1 9 18 20 20 24 32 35 3
Chassis Enters 2
Line 2 12 16 22 22 25 30 37
Station S2,1 Station S2,2 Station S2,3 Station S2,4 Station S2,5 Station S2,6 j j
f1[j] 9 18 20 24 32
35+3
l1[j] 1 2 1 1 2
f*=38
l* =1
f2[j] 12 16 22 25 30
37+2
l2[j] 1 2 1 2 2

28. Step 2: A recursive solution (cont.)
fi[j] values give the values of optimal solutions to subproblems.
Let li[j] to be the line number 1 or 2, whose station j-1 is used in a fw throgh station Si, j.
We also define l* to be the line whose station n is used in a fw through the entire factory.

29. Step 2: A recursive solution (cont.)
The fw through the entire factory:

30. Step 3: Computing the fastest times
We can write a simple recursive algorithm based on the equations for f1[j] and f2[j] to compute the fw through the factory. However its running time is exponential.
Let ri( j ) be the number of references made to fi [ j ] in a recursive algorithm:

31. Step 3: Computing the fastest times(cont.)
We observe that for each value of fi[j] depends only on the values of f1[j-1] and f2[j-1].
By computing the fi[j] values in order of increasing station numbers we can compute the fw through the factory in time.

32. The fastest way procedure

33. The fastest way procedure (cont.)
line 1, station 6
line 2, station 5
line 2, station 4
line 1, station 3
line 2, station 2
line 1, station 1

34. Matrix-Chain multiplication
We are given a sequence
And we wish to compute

35. Matrix-Chain multiplication (cont.)
Matrix multiplication is assosiative, and so all parenthesizations yield the same product.
For example, if the chain of matrices is then the product can be fully paranthesized in five distinct way:

36. Matrix-Chain multiplication
MATRIX-MULTIPLY (A,B)
if columns [A] ≠ rows [B]
then error “incompatible dimensions”
else for i←1 to rows [A]
do for j←1 to columns [B]
do C[i, j]←0
for k←1 to columns [A]
do C[ i, j ]← C[ i, j] +A[ i, k]*B[ k, j]
return C

37. Matrix-Chain multiplication (cont.)
Cost of the matrix multiplication:
An example:

38. Matrix-Chain multiplication (cont.)

39. Matrix-Chain multiplication (cont.)
The problem:
Given a chain of n matrices, where matrix Ai has dimension pi-1x pi, fully paranthesize the product
in a way that minimizes the number of scalar multiplications.

40. Matrix-Chain multiplication (cont.)
Counting the number of alternative paranthesization : bn

41. Matrix-Chain multiplication (cont.)
Based on the above defination we have

42. Matrix-Chain multiplication (cont.)
Substituting the initial value for b we obtain

43. Matrix-Chain multiplication (cont.)
Let B(x) be the generating function

44. Matrix-Chain multiplication (cont.)
Substituting in B(x) we can write that
B(x)

45. Matrix-Chain multiplication (cont.)
Summing over the both sides

46. Matrix-Chain multiplication (cont.)
One way to express B(x) in closed form is
Expanding into a Taylor series about x=0 we get
Comparing the coefficients of x we finally show that

47. Matrix-Chain multiplication (cont.)
Step 1: The structure of an optimal paranthesization(op)
Find the optimal substructure and then use it to construct an optimal solution to the problem from optimal solutions to subproblems.
Let Ai...j where i ≤ j, denote the matrix product Ai Ai Aj
Any parenthesization of Ai Ai Aj must split the product between Ak and Ak+1 for i ≤ k < j.

48. Matrix-Chain multiplication (cont.)
The optimal substructure of the problem:
Suppose that an op of Ai Ai Aj splits the product between Ak and Ak+1 then the paranthesization of the subchain Ai Ai Ak within this parantesization of Ai Ai Aj must be an op of Ai Ai Ak

49. Matrix-Chain multiplication (cont.)
Step 2: A recursive solution:
Let m[i,j] be the minimum number of scalar multiplications needed to compute the matrix Ai...j where 1≤ i ≤ j ≤ n.
Thus, the cost of a cheapest way to compute A1...n would be m[1,n].
Assume that the op splits the product Ai...j between Ak and Ak+1.where i ≤ k <j.
Then m[i,j] =The minimum cost for computing Ai...k and Ak+1...j + the cost of multiplying these two matrices.

50. Matrix-Chain multiplication (cont.)
Recursive defination for the minimum cost of paranthesization:

51. Matrix-Chain multiplication (cont.)
To help us keep track of how to constrct an optimal solution we define s[ i,j] to be a value of k at which we can split the product Ai...j to obtain an optimal paranthesization.
That is s[ i,j] equals a value k such that

52. Matrix-Chain multiplication (cont.)
Step 3: Computing the optimal costs
It is easy to write a recursive algorithm based on recurrence for computing m[i,j].
But the running time will be exponential!...

53. Matrix-Chain multiplication (cont.)
Step 3: Computing the optimal costs
We compute the optimal cost by using a tabular, bottom-up approach.

54. Matrix-Chain multiplication (Contd.)
MATRIX-CHAIN-ORDER(p)
n←length[p]-1
for i←1 to n
do m[i,i]←0
for l←2 to n
do for i←1 to n-l+1
do j←i+l-1
m[i,j]← ∞
for k←i to j-1
do q←m[i,k] + m[k+1,j]+pi-1 pk pj
if q < m[i,j]
then m[i,j] ←q s[i,j] ←k
return m and s

55. Matrix-Chain multiplication (cont.)
An example: matrix dimension
A1 30 x 35
A2 35 x 15
A3 15 x 5
A x 10
A5 10 x 20
A6 20 x 25

56. Matrix-Chain multiplication (cont.)s m 6 1 3 2 6 1 5 i j
15125 2 5 4 3 3 3 i j
10500 3
11875 3 3 4 4 3 3
7125
575 4 5 5
9375 3 3 2 1 3
4375
2500
2500
3500 5 4 5
7875 1 2 3 2
1000
5000
2625
750
1000 6
15750 1 A1 A2 A3 A4 A5 A6

57. Matrix-Chain multiplication (cont.)
Step 4: Constructing an optimal solution
An optimal solution can be constructed from the computed information stored in the table s[1...n, 1...n].
We know that the final matrix multiplication is
The earlier matrix multiplication can be computed recursively.

58. Matrix-Chain multiplication (Contd.)
PRINT-OPTIMAL-PARENS (s, i, j)
1 if i=j
then print “Ai”
else print “ ( “
PRINT-OPTIMAL-PARENS (s, i, s[i,j])
PRINT-OPTIMAL-PARENS (s, s[i,j]+1, j)
Print “ ) ”

59. Matrix-Chain multiplication (Contd.)
RUNNING TIME:
Recursive solution takes exponential time.
Matrix-chain order yields a running time of O(n3)

60. Elements of dynamic programming
When should we apply the method of Dynamic Programming?
Two key ingredients:
- Optimal substructure
- Overlapping subproblems

61. Elements of dynamic programming (cont.)
Optimal substructure (os):
A problem exhibits os if an optimal solution to the problem contains within it optimal solutions to subproblems.
Whenever a problem exhibits os, it is a good clue that dynamic programming might apply.
In dynamic programming, we build an optimal solution to the problem from optimal solutions to subproblems.
Dynamic programming uses optimal substructure in a bottom-up fashion.

62. Elements of dynamic programming (cont.)
Overlapping subproblems:
When a recursive algorithm revisits the same problem over and over again, we say that the optimization problem has overlapping subproblems.
In contrast , a divide-and-conquer approach is suitable usually generates brand new problems at each step of recursion.
Dynamic programming algorithms take advantage of overlapping subproblems by solving each subproblem once and then storing the solution in a table where it can be looked up when needed.

63. Elements of dynamic programming (cont.)
Overlapping subproblems: (cont.)
1..4
1..1
2..4
1..2
3..4
1..3
4..4
2..2
3..4
2..3
4..4
1..1
2..2
3..3
4..4
1..1
2..3
1..2
3..3
2..2
3..3
1..1
2..2
3..3
4..4
2..2
3..3
The recursion tree of RECURSIVE-MATRIX-CHAIN( p, 1, 4). The computations performed in a shaded subtree are replaced by a single table lookup in MEMOIZED-MATRIX-CHAIN( p, 1, 4).

64. Matrix-Chain multiplication (Contd.)
RECURSIVE-MATRIX-CHAIN (p, i, j)
1 if i = j
then return 0
3 m[i,j] ←∞
4 for k←i to j-1
do q←RECURSIVE-MATRIX-CHAIN (p, i, k)
+ RECURSIVE-MATRIX-CHAIN (p, k+1, j)+ pi-1 pk pj
if q < m[i,j]
then m[i,j] ←q
8 return m[i,j]

65. Elements of dynamic programming (cont.)
Overlapping subproblems: (cont.)
Time to compute m[ i,j] by RECURSIVE-MATRIX-CHAIN:
We assume that the execution of lines 1-2 and 6-7 take at least unit time.

66. Elements of dynamic programming (cont.)
Overlapping subproblems: (cont.)
WE guess that
Using the substitution method with

67. Elements of dynamic programming (cont.)
Memoization
There is a variation of dynamic programming that often offers the efficiency of the usual dynamic-programming approach while maintaining a top-down strategy.
The idea is to memoize the the natural, but inefficient, recursive algorithm.
We maintain a table with subproblem solutions, but the control structure for filling in the table is more like the recursive algorithm.

68. Elements of dynamic programming (cont.)
Memoization (cont.)
An entry in a table for the solution to each subproblem is maintained.
Eech table entry initially contains a special value to indicate that the entry has yet to be filled.
When the subproblem is first encountered during the execution of the recursive algorithm, its solution is computed and then stored in the table.
Each subsequent time that the problem is encountered, the value stored in the table is simply looked up and returned.

69. Elements of dynamic programming (cont.)
1 MEMOIZED-MATRIX-CHAIN(p)
2 n←length[p]-1
3 for i←1 to n
do for j←i to n
do m[i,j] ←∞
return LOOKUP-CHAIN(p,1,n)

70. Elements of dynamic programming (cont.)
Memoization (cont.)
LOOKUP-CHAIN(p,1,n)
if m[i,j] < ∞
then return m[i,j]
if i=j
then m[i,j] ←0
else for k←1 to j-1
do q← LOOKUP-CHAIN(p,i,k)
+ LOOKUP-CHAIN(p,k+1,j) + pi-1 pk pj
if q < m[i,j]
then m[i,j] ←q
return m[i,j]

71. Longest Common Subsequence

72. Elements of dynamic programming (cont.)

73. Elements of dynamic programming (cont.)

74. Elements of dynamic programming (cont.)

75. Elements of dynamic programming (cont.)

76. Elements of dynamic programming (cont.)

77. Elements of dynamic programming (cont.)

78. Elements of dynamic programming (cont.)

79. Elements of dynamic programming (cont.)

80. Elements of dynamic programming (cont.)

81. Elements of dynamic programming (cont.)

82. Elements of dynamic programming (cont.)