1. Algorithm Design Techniques: Dynamic Programming

2. Introduction Dynamic Programming –Divide-and-conquer solution to a problem with overlapping subproblems –Similar to bottom-up recursion where results are saved in a table as they are computed –Typically applied to optimization problems –Typically best used when objects have a linear ordering and cannot be rearranged http://www.cs.sunysb.edu/~algorith/lectures-good/node13.html http://www.cs.sunysb.edu/~algorith/lectures-good/node12.html

3. Example: Fibonacci Basic recursive solution F(N) = F(N-1)+F(N-2) F1 calculated three times Instead, calculate numbers 1->n, store values as they are calculated Use space instead of time F4 F3 F2 F1 F0 F1F0

4. Steps (CLR pg323) 1.Characterize the structure of an optimal solution (optimal substructure) 2.Recursively define the value of an optimal solution 3.Compute the value of an optimal solution in a bottom-up fashion 4.Construct an optimal solution from computed information

5. Ordering Matrix Multiplications Matrices –A (50x10), B (10x40), C (40x30), D (30x5) Compute ABCD –Associative (not commutative) Fits the ordering property –Decide how to parenthesize to achieve fewest operations

6. Ordering Matrix Multiplications (A((BC)D) –BC = 10x40x30 = 12,000 operations –(BC)D = 12,000 + 10x30x5 = 13,500 –A((BC)D) = 13,500 + 50x10x5 = 16,000 (A(B(CD)) = 10,500 ((AB)(CD)) = 36,000 (((AB)C)D) = 87,500 ((A(BC))D) = 34,500

7. Ordering Matrix Multiplications N-1 T(N) = Σ T(i)T(N-i) i=1 Basic recursive solution is exponential M Left, Right = min Left<=i<=Right {M Left,i +M i+1,Right +c Left-1 c i c Right }

8. Ordering Matrix Multiplications A B CD A = 50x10 B = 10x40 C = 40x30 D = 30x5 20,00012,0006,000 27,0008,000 10,500 0000 N 2 /2 values are calculated O(N 3 ) running time

9. Optimal Binary Search Tree Create binary search tree to minimize N Σ p i (1+d i ) i=1 WordProbability a.22 am.18 and.20 egg.05 if.25 the.02 two.08 if atwo theand amegg

10. Optimal Binary Search Tree Basic greedy strategy does not work But, problem has the ordering property and optimal substructure property i-1 Right C Left, Right = min{p i + C Left, i-1 +C i+1, Right +  p j +  p j } j=Left j=i+1

11. Optimal Binary Search Tree (pg431) a..a.22 a am..am.18 am and..and.20 and egg..egg.05 egg if..if.25 if the..the.02 the two..two.08 two a..am.58 a am..and.56 am and..egg.30 and egg..if.35 if if..the.29 if the..two.12 two a..and 1.02 am am..egg.66 and and..if.80 if egg..the.39 if if..two.46 if a..egg 1.17 am am..if 1.21 and and..the.84 if egg..two.57 if a..if 1.83 and am..the 1.27 and and..two 1.02 if a..the 1.89 and am..two 1.53 and a..two 2.15 and

12. Memoization Maintain top-down recursion, but memoize (keep "memos" of) results allocate array for memo; initialize memo[1] and memo[2] to 1; fib(n) { if memo[n] is not zero, return memo[n]; memo[n] = fib(n-1) + fib(n-2); return memo[n]; } http://www.nist.gov/dads/HTML/memoize.html