1. Dynamic Programming Chapter 15 Highlights Charles Tappert Seidenberg School of CSIS, Pace University

2. What is dynamic programming? Dynamic programming is a method of solving optimization problems by combining the solutions of subproblems Developing these algorithms follows four steps: 1. Characterize the structure of an optimal solution 2. Recursively define the value of an optimal solution 3. Compute the optimal solution, typically bottom-up 4. Construct the path of an optimal solution (if desired)

3. Example – Rod Cutting Problem: Given a rod of length n inches and a table of prices, determine the maximum revenue obtainable by cutting up the rod and selling the pieces Rod cuts are an integral number of inches, cuts are free Price table for rods

4. Example – Rod Cutting Eight possible ways to cut a rod of length 4 (prices shown on top)

5. Example – Rod Cutting The recursion tree for a rod of length 4 The subproblem graph (collapsed tree)

6. Example – Rod Cutting Recursive equation: Bottom-up algorithm – O(n 2 ) from double nesting

7. Example – Rod Cutting Extended bottom-up algorithm obtains path Print solution

8. Example Longest Common Subsequence A string over a finite set S is a sequence of elements of S (Appendix C, p 1184) Given two sequences X and Y, a sequence Z is a common subsequence of X and Y if Z is a subsequence of both X and Y Problem: Find the maximum length common subsequence of X and Y

9. Example Longest Common Subsequence

10. Other Examples from Textbook Matrix-chain multiplication Optimal binary search trees

11. Two key ingredients for dynamic programming to apply to an optimization problem: 1. Optimal substructure (also for greedy algorithms) 2. Overlapping subproblems Bottom-up algorithms usually outperform top- down ones by a constant factor due to less overhead Elements of Dynamic Programming

12. 1. Optimal substructure Occurs when the optimal solution contains within it optimal solutions to subproblems We build an optimal solution to the problem from optimal solutions to subproblems Rod-cutting a rod of size n uses just one subproblem of size n-i Matrix-chain multiplication uses two subproblems, the two sub-chains Elements of Dynamic Programming

13. 2. Overlapping subproblems This occurs when a recursive algorithm revisits the same problem repeatedly The dynamic programming algorithm typically solves each subproblem only once and stores the result The rod-cutting dynamic programming solution reduced an exponential-time recursive algorithm down to quadratic time Elements of Dynamic Programming

14. Longest-common-subsequence (LCS) problem, used in biological applications to compare the DNA of two different organisms In-class exercise solve problem 15.4-1 (p 396) finding the string length and the string String matching – LCS

15. Online handwriting recognition (journal article) String matching – handwriting In-class Exercise

16. Textbook chapter-end problem 5 (p 406) See Speech and Language book chapter In-class exercise (time permitting) String matching – spell checking

17. Various applications Speech production models Speech recognition systems Speech sound alignment for speaker verification (voiceprint) systems String matching – speech & language

18. Speaker Verification: “My name is …”

19. by two different speakers

20. “My name is” divided into seven sound units. Speaker Verification Alignment Problem: DTW locates the seven sounds

21. Paper (1-3 pages) due last session See link to assignment on Syllabus page String matching – assignment