1. 1 Chapter 6 Dynamic Programming Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved.

2. 2 Algorithmic Paradigms Greedy. Build up a solution incrementally, myopically optimizing some local criterion. Divide-and-conquer. Break up a problem into sub-problems, solve each sub-problem independently, and combine solution to sub-problems to form solution to original problem. Dynamic programming. Break up a problem into a series of overlapping sub-problems, and build up solutions to larger and larger sub-problems.

3. 3 Dynamic Programming History Bellman. [1950s] Pioneered the systematic study of dynamic programming. Etymology. n Dynamic programming = planning over time. n Secretary of Defense was hostile to mathematical research. n Bellman sought an impressive name to avoid confrontation. Reference: Bellman, R. E. Eye of the Hurricane, An Autobiography. "it's impossible to use dynamic in a pejorative sense" "something not even a Congressman could object to"

4. 4 Dynamic Programming Applications Areas. n Bioinformatics. n Control theory. n Information theory. n Operations research. n Computer science: theory, graphics, AI, compilers, systems, …. Some famous dynamic programming algorithms. n Unix diff for comparing two files. n Viterbi for hidden Markov models. n Smith-Waterman for genetic sequence alignment. n Bellman-Ford for shortest path routing in networks. n Cocke-Kasami-Younger for parsing context free grammars.

5. 6.1 Weighted Interval Scheduling

6. 6 Weighted Interval Scheduling Weighted interval scheduling problem. n Job j starts at s j, finishes at f j, and has weight or value v j. n Two jobs compatible if they don't overlap. n Goal: find maximum weight subset of mutually compatible jobs. Time f g h e a b c d 012345678910

7. 7 Unweighted Interval Scheduling Review Recall. Greedy algorithm works if all weights are 1. n Consider jobs in ascending order of finish time. n Add job to subset if it is compatible with previously chosen jobs. Observation. Greedy algorithm can fail spectacularly if arbitrary weights are allowed. Time 01234567891011 b a weight = 999 weight = 1

8. 8 Weighted Interval Scheduling Notation. Label jobs by finishing time: f 1  f 2 ...  f n. Def. p(j) = largest index i < j such that job i is compatible with j. Ex: p(8) = 5, p(7) = 3, p(2) = 0. Time 01234567891011 6 7 8 4 3 1 2 5

9. 9 Dynamic Programming: Binary Choice Notation. OPT(j) = value of optimal solution to the problem consisting of job requests 1, 2,..., j. n Case 1: OPT selects job j. – collect profit v j – can't use incompatible jobs { p(j) + 1, p(j) + 2,..., j - 1 } – must include optimal solution to problem consisting of remaining compatible jobs 1, 2,..., p(j) n Case 2: OPT does not select job j. – must include optimal solution to problem consisting of remaining compatible jobs 1, 2,..., j-1 optimal substructure

10. 10 Input: n, s 1,…,s n, f 1,…,f n, v 1,…,v n Sort jobs by finish times so that f 1  f 2 ...  f n. Compute p(1), p(2), …, p(n) Compute-Opt(j) { if (j = 0) return 0 else return max(v j + Compute-Opt(p(j)), Compute-Opt(j-1)) } Weighted Interval Scheduling: Brute Force Brute force algorithm.

11. 11 Weighted Interval Scheduling: Brute Force Observation. Recursive algorithm fails spectacularly because of redundant sub-problems  exponential algorithms. Ex. Number of recursive calls for family of "layered" instances grows like Fibonacci sequence. 3 4 5 1 2 p(1) = 0, p(j) = j-2 5 43 3221 21 10 1010

12. 12 Input: n, s 1,…,s n, f 1,…,f n, v 1,…,v n Sort jobs by finish times so that f 1  f 2 ...  f n. Compute p(1), p(2), …, p(n) for j = 1 to n M[j] = empty M[0] = 0 M-Compute-Opt(j) { if (M[j] is empty) M[j] = max(v j + M-Compute-Opt(p(j)), M-Compute-Opt(j-1)) return M[j] } global array Weighted Interval Scheduling: Memoization Memoization. Store results of each sub-problem in a cache; lookup as needed.

13. 13 Weighted Interval Scheduling: Running Time Claim. Memoized version of algorithm takes O(n log n) time. n Sort by finish time: O(n log n). n Computing p(  ) : O(n log n) via sorting by start time. M-Compute-Opt(j) : each invocation takes O(1) time and either – (i) returns an existing value M[j] – (ii) fills in one new entry M[j] and makes two recursive calls Progress measure  = # nonempty entries of M[]. – initially  = 0, throughout   n. – (ii) increases  by 1  at most 2n recursive calls. Overall running time of M-Compute-Opt(n) is O(n). ▪ Remark. O(n) if jobs are pre-sorted by start and finish times.

14. 14 Weighted Interval Scheduling: Finding a Solution Q. Dynamic programming algorithms computes optimal value. What if we want the solution itself? A. Do some post-processing. n # of recursive calls  n  O(n). Run M-Compute-Opt(n) Run Find-Solution(n) Find-Solution(j) { if (j = 0) output nothing else if (v j + M[p(j)] > M[j-1]) print j Find-Solution(p(j)) else Find-Solution(j-1) }

15. 15 Weighted Interval Scheduling: Bottom-Up Bottom-up dynamic programming. Unwind recursion. Input: n, s 1,…,s n, f 1,…,f n, v 1,…,v n Sort jobs by finish times so that f 1  f 2 ...  f n. Compute p(1), p(2), …, p(n) Iterative-Compute-Opt { M[0] = 0 for j = 1 to n M[j] = max(v j + M[p(j)], M[j-1]) }

16. 6.3 Segmented Least Squares

17. 17 Segmented Least Squares Least squares. n Foundational problem in statistic and numerical analysis. n Given n points in the plane: (x 1, y 1 ), (x 2, y 2 ),..., (x n, y n ). n Find a line y = ax + b that minimizes the sum of the squared error: Solution. Calculus  min error is achieved when x y

18. 18 Segmented Least Squares Segmented least squares. n Points lie roughly on a sequence of several line segments. n Given n points in the plane (x 1, y 1 ), (x 2, y 2 ),..., (x n, y n ) with n x 1 < x 2 <... < x n, find a sequence of lines that minimizes f(x). Q. What's a reasonable choice for f(x) to balance accuracy and parsimony? x y goodness of fit number of lines

19. 19 Segmented Least Squares Segmented least squares. n Points lie roughly on a sequence of several line segments. n Given n points in the plane (x 1, y 1 ), (x 2, y 2 ),..., (x n, y n ) with n x 1 < x 2 <... < x n, find a sequence of lines that minimizes: – the sum of the sums of the squared errors E in each segment – the number of lines L n Tradeoff function: E + c L, for some constant c > 0. x y

20. 20 Dynamic Programming: Multiway Choice Notation. n OPT(j) = minimum cost for points p 1, p i+1,..., p j. n e(i, j) = minimum sum of squares for points p i, p i+1,..., p j. To compute OPT(j): n Last segment uses points p i, p i+1,..., p j for some i. n Cost = e(i, j) + c + OPT(i-1).

21. 21 Segmented Least Squares: Algorithm Running time. O(n 3 ). n Bottleneck = computing e(i, j) for O(n 2 ) pairs, O(n) per pair using previous formula. INPUT: n, p 1,…,p N, c Segmented-Least-Squares() { M[0] = 0 for j = 1 to n for i = 1 to j compute the least square error e ij for the segment p i,…, p j for j = 1 to n M[j] = min 1  i  j (e ij + c + M[i-1]) return M[n] } can be improved to O(n 2 ) by pre-computing various statistics

22. 6.4 Knapsack Problem

23. 23 Knapsack Problem Knapsack problem. n Given n objects and a "knapsack." n Item i weighs w i > 0 kilograms and has value v i > 0. n Knapsack has capacity of W kilograms. n Goal: fill knapsack so as to maximize total value. Ex: { 3, 4 } has value 40. Greedy: repeatedly add item with maximum ratio v i / w i. Ex: { 5, 2, 1 } achieves only value = 35  greedy not optimal. 1 value 18 22 28 1 weight 5 6 62 7 # 1 3 4 5 2 W = 11

24. 24 Dynamic Programming: False Start Def. OPT(i) = max profit subset of items 1, …, i. n Case 1: OPT does not select item i. – OPT selects best of { 1, 2, …, i-1 } n Case 2: OPT selects item i. – accepting item i does not immediately imply that we will have to reject other items – without knowing what other items were selected before i, we don't even know if we have enough room for i Conclusion. Need more sub-problems!

25. 25 Dynamic Programming: Adding a New Variable Def. OPT(i, w) = max profit subset of items 1, …, i with weight limit w. n Case 1: OPT does not select item i. – OPT selects best of { 1, 2, …, i-1 } using weight limit w n Case 2: OPT selects item i. – new weight limit = w – w i – OPT selects best of { 1, 2, …, i–1 } using this new weight limit

26. Knapsack. Fill up an n-by-W array. 26 Input: n, W, w 1,…,w N, v 1,…,v N for w = 0 to W M[0, w] = 0 for i = 1 to n for w = 1 to W if (w i > w) M[i, w] = M[i-1, w] else M[i, w] = max {M[i-1, w], v i + M[i-1, w-w i ]} return M[n, W] Knapsack Problem: Bottom-Up

27. 27 Knapsack Algorithm n + 1 1 Value 18 22 28 1 Weight 5 6 62 7 Item 1 3 4 5 2  { 1, 2 } { 1, 2, 3 } { 1, 2, 3, 4 } { 1 } { 1, 2, 3, 4, 5 } 0 0 0 0 0 0 0 1 0 1 1 1 1 1 2 0 6 6 6 1 6 3 0 7 7 7 1 7 4 0 7 7 7 1 7 5 0 7 18 1 6 0 7 19 22 1 7 0 7 24 1 28 8 0 7 25 28 1 29 9 0 7 25 29 1 34 10 0 7 25 29 1 34 11 0 7 25 40 1 W + 1 W = 11 OPT: { 4, 3 } value = 22 + 18 = 40

28. 28 Knapsack Algorithm: Finding a Solution Q. Knapsack algorithms computes optimal value. What if we want the solution itself? A. Do some post-processing (similar to weighted interval scheduling). n Start from the last item i=n, and w=W, Considering M[n, W]: If M[i, w] = M[i-1, w], then OPT does not include item i. – We move to consider item M[i-1, w] n If M[i, w] = v i + M[i-1, w-w i ], then OPT includes current item i.  output item i – We move to consider item M[i-1, w-w i ] n # of recursive calls  n  O(n).

29. 29 Knapsack Problem: Running Time Running time.  (n W). n Not polynomial in input size! n "Pseudo-polynomial." n Decision version of Knapsack is NP-complete. [Chapter 8] Knapsack approximation algorithm. There exists a poly-time algorithm that produces a feasible solution that has value within 0.01% of optimum. [Section 11.8]

30. 6.5 RNA Secondary Structure

31. 31 RNA Secondary Structure RNA. String B = b 1 b 2  b n over alphabet { A, C, G, U }. Secondary structure. RNA is single-stranded so it tends to loop back and form base pairs with itself. This structure is essential for understanding behavior of molecule. G U C A GA A G CG A U G A U U A G A CA A C U G A G U C A U C G G G C C G Ex: GUCGAUUGAGCGAAUGUAACAACGUGGCUACGGCGAGA complementary base pairs: A-U, C-G

32. 32 RNA Secondary Structure Secondary structure. A set of pairs S = { (b i, b j ) } that satisfy: n [Watson-Crick.] S is a matching and each pair in S is a Watson- Crick complement: A-U, U-A, C-G, or G-C. n [No sharp turns.] The ends of each pair are separated by at least 4 intervening bases. If (b i, b j )  S, then i < j - 4. n [Non-crossing.] If (b i, b j ) and (b k, b l ) are two pairs in S, then we cannot have i < k < j < l. Free energy. Usual hypothesis is that an RNA molecule will form the secondary structure with the optimum total free energy. Goal. Given an RNA molecule B = b 1 b 2  b n, find a secondary structure S that maximizes the number of base pairs. approximate by number of base pairs

33. 33 RNA Secondary Structure: Examples Examples. C GG C A G U U UA A UGUGGCCAU GG C A G U UA A UGGGCAU C GG C A U G U UA A GUUGGCCAU sharp turncrossingok G G 44 base pair

34. 34 RNA Secondary Structure: Subproblems First attempt. OPT(j) = maximum number of base pairs in a secondary structure of the substring b 1 b 2  b j. Difficulty. Results in two sub-problems. n Finding secondary structure in: b 1 b 2  b t-1. n Finding secondary structure in: b t+1 b t+2  b n-1. 1 tn match b t and b n OPT(t-1) need more sub-problems

35. 35 Dynamic Programming Over Intervals Notation. OPT(i, j) = maximum number of base pairs in a secondary structure of the substring b i b i+1  b j. n Case 1. If i  j - 4. – OPT(i, j) = 0 by no-sharp turns condition. n Case 2. Base b j is not involved in a pair. – OPT(i, j) = OPT(i, j-1) n Case 3. Base b j pairs with b t for some i  t < j - 4. – non-crossing constraint decouples resulting sub-problems – OPT(i, j) = 1 + max t { OPT(i, t-1) + OPT(t+1, j-1) } take max over t such that i  t < j-4 and b t and b j are Watson-Crick complements

36. 36

37. 37 Bottom Up Dynamic Programming Over Intervals Q. What order to solve the sub-problems? A. Do shortest intervals first. Running time. O(n 3 ). RNA(b 1,…,b n ) { for k = 5, 6, …, n-1 for i = 1, 2, …, n-k j = i + k Compute M[i, j] return M[1, n] } using recurrence OPT(i, j) = max{ OPT(i, j-1), 1 + max t { OPT(i, t-1) + OPT(t+1, j-1) } } 000 00 0 2 3 4 1 i 6789 j take max over t such that i  t < j-4 and b t and b j are Watson-Crick complements

38. 38 RNA(b 1,…,b n ) { for k = 5, 6, …, n-1 for i = 1, 2, …, n-k j = i + k Compute M[i, j] return M[1, n] }

39. 39 Dynamic Programming Summary Recipe. n Characterize structure of problem. n Recursively define value of optimal solution. n Compute value of optimal solution. n Construct optimal solution from computed information. Dynamic programming techniques. n Binary choice: weighted interval scheduling. n Multi-way choice: segmented least squares. n Adding a new variable: knapsack. n Dynamic programming over intervals: RNA secondary structure. Top-down vs. bottom-up: different people have different intuitions. Viterbi algorithm for HMM also uses DP to optimize a maximum likelihood tradeoff between parsimony and accuracy CKY parsing algorithm for context-free grammar has similar structure

40. 6.6 Sequence Alignment

41. 41 String Similarity How similar are two strings? n ocurrance n occurrence ocurrance ccurrenceo - ocurrnce ccurrnceo --a e- ocurrance ccurrenceo - 6 mismatches, 1 gap 1 mismatch, 1 gap 0 mismatches, 3 gaps

42. 42 Applications. n Basis for Unix diff. n Speech recognition. n Computational biology. Edit distance. [Levenshtein 1966, Needleman-Wunsch 1970] n Gap penalty  ; mismatch penalty  pq. n Cost = sum of gap and mismatch penalties. 2  +  CA CGACCTACCT CTGACTACAT TGACCTACCT CTGACTACAT - T C C C  TC +  GT +  AG + 2  CA - Edit Distance

43. 43 Goal: Given two strings X = x 1 x 2... x m and Y = y 1 y 2... y n find alignment of minimum cost. Def. An alignment M is a set of ordered pairs x i -y j such that each item occurs in at most one pair and no crossings. Def. The pair x i -y j and x i' -y j' cross if i j'. Ex: CTACCG vs. TACATG. Sol: M = x 2 -y 1, x 3 -y 2, x 4 -y 3, x 5 -y 4, x 6 -y 6. Sequence Alignment CTACC- TACAT- G G y1y1 y2y2 y3y3 y4y4 y5y5 y6y6 x2x2 x3x3 x4x4 x5x5 x1x1 x6x6

44. 44 Sequence Alignment: Problem Structure Def. OPT(i, j) = min cost of aligning strings x 1 x 2... x i and y 1 y 2... y j. n Case 1: OPT matches x i -y j. – pay mismatch for x i -y j + min cost of aligning two strings x 1 x 2... x i-1 and y 1 y 2... y j-1 n Case 2a: OPT leaves x i unmatched. – pay gap for x i and min cost of aligning x 1 x 2... x i-1 and y 1 y 2... y j n Case 2b: OPT leaves y j unmatched. – pay gap for y j and min cost of aligning x 1 x 2... x i and y 1 y 2... y j-1

45. 45 Sequence Alignment: Algorithm Analysis.  (mn) time and space. English words or sentences: m, n  10. Computational biology: m = n = 100,000. 10 billions ops OK, but 10GB array? Sequence-Alignment(m, n, x 1 x 2...x m, y 1 y 2...y n, ,  ) { for i = 0 to m M[i, 0] = i  for j = 0 to n M[0, j] = j  for i = 1 to m for j = 1 to n M[i, j] = min(  [x i, y j ] + M[i-1, j-1],  + M[i-1, j],  + M[i, j-1]) return M[m, n] }

46. 6.7 Sequence Alignment in Linear Space

47. 47 Sequence Alignment: Linear Space Q. Can we avoid using quadratic space? Easy. Optimal value in O(m + n) space and O(mn) time. n Compute OPT(i, ) from OPT(i-1, ). n No longer a simple way to recover alignment itself. Theorem. [Hirschberg 1975] Optimal alignment in O(m + n) space and O(mn) time. n Clever combination of divide-and-conquer and dynamic programming. n Inspired by idea of Savitch from complexity theory.

48. Observation: Comput M[i,j] only needs M[i,1,*] and M[i,*] Conclusion: Don’t need m*n matrix, just use m*2 matrix 48 Sequence-Alignment(m, n, x 1 x 2...x m, y 1 y 2...y n, ,  ) { for i = 0 to m M[i, 0] = i  for j = 0 to n M[0, j] = j  for i = 1 to m for j = 1 to n M[i, j] = min(  [x i, y j ] + M[i-1, j-1],  + M[i-1, j],  + M[i, j-1]) return M[m, n] }

49. 49 Edit distance graph. n Let f(i, j) be shortest path from (0,0) to (i, j). n Observation: f(i, j) = OPT(i, j). Sequence Alignment: Linear Space i-j m-n x1x1 x2x2 y1y1 x3x3 y2y2 y3y3 y4y4 y5y5 y6y6   0-0  

50. 50 Edit distance graph. n Let f(i, j) be shortest path from (0,0) to (i, j). Sequence Alignment: Linear Space i-j m-n x1x1 x2x2 y1y1 x3x3 y2y2 y3y3 y4y4 y5y5 y6y6   0-0 j

51. 51 Edit distance graph. n Let g(i, j) be shortest path from (i, j) to (m, n). n Can compute by reversing the edge orientations and inverting the roles of (0, 0) and (m, n) Sequence Alignment: Linear Space i-j m-n x1x1 x2x2 y1y1 x3x3 y2y2 y3y3 y4y4 y5y5 y6y6   0-0  

52. 52 Edit distance graph. n Let g(i, j) be shortest path from (i, j) to (m, n). Sequence Alignment: Linear Space i-j m-n x1x1 x2x2 y1y1 x3x3 y2y2 y3y3 y4y4 y5y5 y6y6   0-0 j

53. 53 Observation 1. The cost of the shortest path that uses (i, j) is f(i, j) + g(i, j). Sequence Alignment: Linear Space i-j m-n x1x1 x2x2 y1y1 x3x3 y2y2 y3y3 y4y4 y5y5 y6y6   0-0

54. 54 Observation 2. let q be an index that minimizes f(q, n/2) + g(q, n/2). Then, the shortest path from (0, 0) to (m, n) uses (q, n/2). Sequence Alignment: Linear Space i-j m-n x1x1 x2x2 y1y1 x3x3 y2y2 y3y3 y4y4 y5y5 y6y6   0-0 n / 2 q

55. 55 Divide: find index q that minimizes f(q, n/2) + g(q, n/2) using DP. n Align x q and y n/2. Conquer: recursively compute optimal alignment in each piece. Sequence Alignment: Linear Space i-j x1x1 x2x2 y1y1 x3x3 y2y2 y3y3 y4y4 y5y5 y6y6   0-0 q n / 2 m-n

56. 56