1. Dynamic Programming Optimization Problems Dynamic Programming Paradigm
Example: Matrix multiplication
Principle of Optimality
Example: String Matching Problems

2. Optimization Problems
In an optimization problem, there are typically many feasible solutions for any input instance I
For each solution S, we have a cost or value function f(S)
Typically, we wish to find a feasible solution S such that f(S) is either minimized or maximized
Thus, when designing an algorithm to solve an optimization problem, we must prove the algorithm produces a best possible solution.

3. Example Problem
You have six hours to complete as many tasks as possible, all of which are equally important.
Task A - 2 hours Task D hours
Task B - 4 hours Task E - 2 hours
Task C - 1/2 hour Task F - 1 hour
How many can you get done?
Is this a minimization or a maximization problem?
Give one example of a feasible but not optimal solution along with its associated value.
Give an optimal solution and its associated value.

4. Dynamic Programming
The key idea behind dynamic program is that it is a divide-and-conquer technique at heart
That is, we solve larger problems by patching together solutions to smaller problems
However, dynamic programming is typically faster because we compute these solutions in a bottom-up fashion

5. Fibonacci numbers F(n) = F(n-1) + F(n-2)
Top-down recursive computation is very inefficient
Many F(i) values are computed multiple times
Bottom-up computation is much more efficient
Compute F(2), then F(3), then F(4), etc. using stored values for smaller F(i) values to compute next value
Each F(i) value is computed just once

6. Recursive Computation
F(n) = F(n-1) + F(n-2) ; F(0) = 0, F(1) = 1
Recursive Solution:
F(6) = 8
F(1)
F(0)
F(2)
F(3)
F(4)
F(5)
Implementing it as a recursive procedure is easy but slow!
We keep calculating the same value over and over!

7. Bottom-up computation
We can calculate F(n) in linear time by storing small values.
F[0] = 0
F[1] = 1
for i = 2 to n
F[i] = F[i-1] + F[i-2]
return F[n]
Moral: We can sometimes trade space for time.
Dynamic programming is a technique for efficiently computing recurrences by storing partial results.
Once you understand dynamic programming, it is usually easier to reinvent certain algorithms than try to look them up!
Dynamic programming is best understood by looking at a bunch of different examples.

8. Key implementation steps
Identify subsolutions that may be useful in computing whole solution
Often need to introduce parameters
Develop a recurrence relation (recursive solution)
Set up the table of values/costs to be computed
The dimensionality is typically determined by the number of parameters
The number of values should be polynomial
Determine the order of computation of values
Backtrack through the table to obtain complete solution (not just solution value)

9. Example: Matrix Multiplication
Input
List of n matrices to be multiplied together using traditional matrix multiplication
The dimensions of the matrices are sufficient
Task
Compute the optimal ordering of multiplications to minimize total number of scalar multiplications performed
Observations:
Multiplying an X  Y matrix by a Y  Z matrix takes X  Y  Z multiplications
Matrix multiplication is associative but not commutative

10. Example Input Input: Feasible solutions and their values
M1, M2, M3, M4
M1: 13 x 5
M2: 5 x 89
M3: 89 x 3
M4: 3 x 34
Feasible solutions and their values
((M1 M2) M3) M4:10,582 scalar multiplications
(M1 M2) (M3 M4): 54,201 scalar multiplications
(M1 (M2 M3)) M4: 2856 scalar multiplications
M1 ((M2 M3) M4): 4055 scalar multiplications
M1 (M2 (M3 M4)): 26,418 scalar multiplications

11. Key implementation steps
Identify subsolutions that may be useful in computing whole solution
Often need to introduce parameters
Define dimensions to be (d0, d1, …, dn) where matrix Mi has dimensions di-1 x di
Let M(i,j) be the matrix formed by multiplying matrices Mi through Mj
Define C(i,j) to be the minimum cost for computing M(i,j)

12. Key implementation steps
Develop a recurrence relation (recursive solution)
Recurrence relation for C(i,j)
C(i,j) = mink=i to j-1 ( C(i,k)+ C(k+1,j) + di-1dkdj)
The last multiplication is between matrices M(i,k) and M(k+1,j)
C(i,i) = 0
Set up the table of values/costs to be computed
The dimensionality is typically determined by the number of parameters
The number of values should be polynomial C 1 2 3 4

13. Key implementation steps
Determine the order of computation of values C 1 2 3 4

14. Computing actual ordering1 2 3 4
5785
1530
2856
1335
1845
9078 P 1 2 3 4
P(i,j) records the intermediate multiplication k used to compute
M(i,j). That is, P(i,j) = k if last multiplication was M(i,k) M(k+1,j)

15. Pseudocode int MatrixOrder() forall i, j C[i, j] = 0; for j = 2 to n
for i = j-1 to 1
C(i,j) = mini<=k<=j-1 (C(i,k)+ C(k+1,j) + di-1dkdj)
P[i, j]=k;
return C[1, n];

16. Backtracking Procedure ShowOrder(i, j) if (i=j) write ( “Ai”) ; else
k = P [ i, j ] ;
write “ ( ” ;
ShowOrder(i, k) ;
write “  ” ;
ShowOrder (k+1, j) ;
write “)” ;

17. Principle of Optimality
In book, this is termed “Optimal substructure”
An optimal solution contains within it optimal solutions to subproblems.
More detailed explanation
Suppose solution S is optimal for problem P.
Suppose we decompose P into P1 through Pk and that S can be decomposed into pieces S1 through Sk corresponding to the subproblems.
Then solution Si is an optimal solution for subproblem Pi

18. Example 1 Matrix Multiplication
In our solution for computing matrix M(1,n), we have a final step of multiplying matrices M(1,k) and M(k+1,n).
Our subproblems then would be to compute M(1,k) and M(k+1,n)
Our solution uses optimal solutions for computing M(1,k) and M(k+1,n) as part of the overall solution.

19. Example 2 Shortest Path Problem
Suppose a shortest path from s to t visits u
We can decompose the path into s-u and u-t.
The s-u path must be a shortest path from s to u, and the u-t path must be a shortest path from u to t
Conclusion: dynamic programming can be used for computing shortest paths

20. Example 3 Longest Path Problem Conclusion?
Suppose a longest path from s to t visits u
We can decompose the path into s-u and u-t.
Is it true that the s-u path must be a longest path from s to u?
Conclusion?

21. Example 4: The Traveling Salesman Problem
What recurrence relation will return the optimal solution to the Traveling Salesman Problem?
If T(i) is the optimal tour on the first i points, will this help us in solving larger instances of the problem?
Can we set T(i+1) to be T(i) with the additional point inserted in the position that will result in the shortest path?

22. No!
T(5)
Shortest Tour
T(4)

23. Summary of bad examples
There almost always is a way to have the optimal substructure if you expand your subproblems enough
For longest path and TSP, the number of subproblems grows to exponential size
This is not useful as we do not want to compute an exponential number of solutions

24. When is dynamic programming effective?
Dynamic programming works best on objects that are linearly ordered and cannot be rearranged
characters in a string
files in a filing cabinet
points around the boundary of a polygon
the left-to-right order of leaves in a search tree.
Whenever your objects are ordered in a left-to-right way, dynamic programming must be considered.

25. Efficient Top-Down Implementation
We can implement any dynamic programming solution top-down by storing computed values in the table
If all values need to be computed anyway, bottom up is more efficient
If some do not need to be computed, top-down may be faster

26. Inexact Matching of Strings
General Problem
Input
Strings S and T
Questions
How distant is S from T?
How similar is S to T?
Solution Technique
Dynamic programming with cost/similarity/scoring matrix

27. Measuring Distance of S and T
Consider S and T
We can transform S into T using the following four operations
insertion of a character into S
deletion of a character from S
substitution (replacement) of a character in S by another character (typically in T)
matching (no operation)

28. Example S = vintner T = writers vintner wintner (Replace v with w)
wrintner (Insert r)
writner (Delete first n)
writer (Delete second n)
writers (Insert S)

29. Example Edit Transcript (or just transcript): Example
a string that describes the transformation of one string into the other
Example
RIMDMDMMI
v intner
wri t ers

30. Edit Distance Edit distance of strings S and T Optimal transcript
The minimum number of edit operations (insertion, deletion, replacement) needed to transform string S into string T
Levenshtein distance, Levenshtein appears to have been the first to define this concept
Optimal transcript
An edit transcript of S and T that has the minimum number of edit operations
cooptimal transcripts

31. Alignment A global alignment of strings S and T is obtained
by inserting spaces (dashes) into S and T
they should have the same number of characters (including dashes) at the end
then placing two strings over each other matching one character (or dash) in S with a unique character (or dash) in T
Note ALL positions in both S and T are involved

32. Alignments and Edit transcripts
Example Alignment
v-intner-
wri-t-ers
Alignments and edit transcripts are interrelated
edit transcript: emphasizes process
the specific mutational events
alignment: emphasizes product
the relationship between the two strings
Alignments are often easier to work with and visualize
also generalize better to more than 2 strings

33. Edit Distance Problem Input Task Solution method 2 strings S and T
Output edit distance of S and T
Output optimal edit transcript
Output optimal alignment
Solution method
Dynamic Programming

34. Identifying Subproblems
Let D(i,j) be the edit distance of S[1..i] and T[1..j]
The edit distance of the first i characters of S with the first j characters of T
Let |S| = n, |T| = m
D(n,m) = edit distance of S and T
We will compute D(i,j) for all i and j such that 0 <= i <= n, 0 <= j <= m

35. Recurrence Relation Base Case: Recursive Case:
For 0 <= i <= n, D(i,0) = i
For 0 <= j <= m, D(0,j) = j
Recursive Case:
0 < i <= n, 0 < j <= m
D(i,j) = min
D(i-1,j) + 1 (what does this mean?)
D(i,j-1) + 1 (what does this mean?)
D(i-1,j-1) + d(i,j) (what does this mean?)
d(i,j) = 0 if S(i) = T(j) and is 1 otherwise

36. What the various cases mean
D(i,j) = min
D(i-1,j) + 1:
Align S[1..i-1] with T[1..j] optimally
Match S(i) with a dash in T
D(i,j-1) + 1
Align S[1..i] with T[1..j-1] optimally
Match a dash in S with T(j)
D(i-1,j-1) + d(i,j)
Align S[1..i-1] with T[1..j-1] optimally
Match S(i) with T(j)

37. Computing D(i,j) valuesw r i t e s 1 2 3 4 5 6 7 v n

38. Initialization: Base Case
D(i,j) w r i t e s 1 2 3 4 5 6 7 v n

39. Row i=1
D(i,j) w r i t e s 1 2 3 4 5 6 7 v n

40. Entry i=2, j=2
D(i,j) w r i t e s 1 2 3 4 5 6 7 v ? n

41. Entry i=2, j=3
D(i,j) w r i t e s 1 2 3 4 5 6 7 v ? n

42. Calculation methodologies
Location of edit distance
D(n,m)
Example was to calculate row by row
Can also calculate column by column
Can also use antidiagonals
Key is to build from upper left corner

43. Traceback Using table to construct optimal transcript
Pointers in cell D(i,j)
Set a pointer from cell (i,j) to
cell (i, j-1) if D(i,j) = D(i, j-1) + 1
cell (i-1,j) if D(i,j) = D(i-1,j) + 1
cell (i-1,j-1) if D(i,j) = D(i-1,j-1) + d(i,j)
Follow path of pointers from (n,m) back to (0,0)

44. What the pointers mean horizontal pointer: cell (i,j) to cell (i, j-1)
Align T(j) with a space in S
Insert T(j) into S
vertical pointer: cell (i,j) to cell (i-1, j)
Align S(i) with a space in T
Delete S(i) from S
diagonal pointer: cell (i,j) to cell (i-1, j-1)
Align S(i) with T(j)
Replace S(i) with T(j)

45. Table and transcripts The pointers represent all optimal transcripts
Theorem:
Any path from (n,m) to (0,0) following the pointers specifies an optimal transcript.
Conversely, any optimal transcript is specified by such a path.
The correspondence between paths and transcripts is one to one.

46. Running Time Initialization of table Calculating table and pointers
O(n+m)
Calculating table and pointers
O(nm)
Traceback for one optimal transcript or optimal alignment

47. Operation-Weight Edit Distance
Consider S and T
We can assign weights to the various operations
insertion/deletion of a character: cost d
substitution (replacement) of a character: cost r
matching: cost e
Previous case: d = r = 1, e = 0

48. Modified Recurrence Relation
Base Case:
For 0 <= i <= n, D(i,0) = i d
For 0 <= j <= m, D(0,j) = j d
Recursive Case:
0 < i <= n, 0 < j <= m
D(i,j) = min
D(i-1,j) + d
D(i,j-1) + d
D(i-1,j-1) + d(i,j)
d(i,j) = e if S(i) = T(j) and is r otherwise

49. Alphabet-Weight Edit Distance
Define weight of each possible substitution
r(a,b) where a is being replaced by b for all a,b in the alphabet
For example, with DNA, maybe r(A,T) > r(A,G)
Likewise, I(a) may vary by character
Operation-weight edit distance is a special case of this variation
Weighted edit distance refers to this alphabet-weight setting

50. Modified Recurrence Relation
Base Case:
For 0 <= i <= n, D(i,0) = S1 <= k <= i I(S(k))
For 0 <= j <= m, D(0,j) = S1 <= k <= j I(T(k))
Recursive Case:
0 < i <= n, 0 < j <= m
D(i,j) = min
D(i-1,j) + I(S(i))
D(i,j-1) + I(T(j))
D(i-1,j-1) + d(i,j)
d(i,j) = r(S(i), T(j))

51. Measuring Similarity of S and T
Definitions
Let S be the alphabet for strings S and T
Let S’ be the alphabet S with character - added
For any two characters x,y in S’, s(x,y) denotes the value (or score) obtained by aligning x with y
For a given alignment A of S and T, let S’ and T’ denote the strings after the chosen insertion of spaces and l their new length
The value of alignment A is S1<=i<=l s(S’(i),T’(i))

52. Example s a b - 1 -2 2 -1 a b a a - b a b a a a a a b - b2 -1
a b a a - b a b
a a a a a b - b =5

53. String Similarity Problem
Input
2 strings S and T
Scoring matrix s for alphabet S’
Task
Output optimal alignment value of S and T
The alignment of S and T with maximal, not minimal, value
Output this alignment

54. Modified Recurrence Relation
Base Case:
For 0 <= i <= n, V(i,0) = S1 <= k <= i s(S(k),-)
For 0 <= j <= m, V(0,j) = S1 <= k <= j s(-,T(k))
Recursive Case:
0 < i <= n, 0 < j <= m
V(i,j) = max
V(i-1,j) + s(S(i),-)
V(i,j-1) + s(-,T(j))
V(i-1,j-1) + s(S(i), T(j))

55. Longest Common Subsequence Problem
Given 2 strings S and T, a common subsequence is a subsequence that appears in both S and T.
The longest common subsequence problem is to find a longest common subsequence (lcs) of S and T
subsequence: characters need not be contiguous
different than substring
Can you use dynamic programming to solve the longest common subsequence problem?

56. Computing alignments using linear space.
Hirschberg [1977]
Suppose we only need the maximum similarity/distance value of S and T without an alignment or transcript
How can we conserve space?
Only save row i-1 when computing row i in the table

57. Illustration
… m 1 2 3 4 . . .
n-1 n

58. Linear space and an alignment
Assume S has length 2n
Divide and conquer approach
Compute value of optimal alignment of S[1..n] with all prefixes of T
Store row n only at end along with pointer values of row n
Compute value of optimal alignment of Sr[1..n] with all prefixes of Tr
Store only values in row n
Find k such that
V(S[1..n],T[1..k]) + V(Sr[1..n],Tr[1..m-k])
is maximized over 0 <= k <=m

59. Illustration
k=0 1 2 3 4 5 6
V(S[1..6],
T[1..0])
m-k=18
V(Sr[1..6],
Tr[1..18]) 6 5 4 3 2 1

60. Illustration
k=1 1 2 3 4 5 6
V(S[1..6],
T[1..1])
m-k=17
V(Sr[1..6],
Tr[1..17]) 6 5 4 3 2 1

61. Illustration
k=2 1 2 3 4 5 6
V(S[1..6],
T[1..2])
m-k=16
V(Sr[1..6],
Tr[1..16]) 6 5 4 3 2 1

62. Illustration
k=9 1 2 3 4 5 6
V(S[1..6],
T[1..9])
m-k=9
V(Sr[1..6],
Tr[1..9]) 6 5 4 3 2 1

63. Illustration
k=18 1 2 3 4 5 6
V(S[1..6],
T[1..18])
m-k=0
V(Sr[1..6],
Tr[1..0]) 6 5 4 3 2 1

64. Illustration 1 2 3 4 5 6 6 5 4 3 2 1

65. Recursive Step Let k* be the k that maximizes
V(S[1..n],T[1..k]) + V(Sr[1..n],Tr[1..m-k])
Record all steps on row n including the one from n-1 and the one to n+1
Recurse on the two subproblems
S[1..n-1] with T[1..j] where j <= k*
Sr[1..n] with Tr[1..q] where q <= m-k*

66. Illustration 1 2 3 4 5 6 6 5 4 3 2 1

67. Time Required cmn time to get this answer so far
Two subproblems have at most half the total size of this problem
At most the same cmn time to get the rest of the solution
cmn/2 + cmn/4 + cmn/8 + cmn/16 + … <= cmn
Final result
Linear space with only twice as much time