1. Advanced Programming Techniques
Divide and Conquer
Dynamic Programming

2. Divide and Conquer You’ve already done this… Binary search Merge sort
Quicksort

3. Exponentiation Brute force O(n): public int pow(int b, int e) {
int result = 1;
for(int n = 0; n < e; n++)
result *= b;
return result; }

4. Exponentiation

5. Comparison
O(n)
O(lgn)

6. Idea…
2  2x2 = 4
4  4x4 = 16
16  16x16 = 256
Done.

7. Exponentiation - recursive
public int pow(int b, int e) { if(e == 0) return 1; else if(e % 2 == 0) return pow(b, e/2) * pow(b, e/2); else//odd return b * pow(b, e-1);//even }

8. Exponentiation - loop
public int pow(int b, int e) { int result = 1; while(e > 0) if(e % 2 == 1)//odd result = result * b; e /= 2; b *= b; } return result;

9. Dynamic Programming
In a divide and conquer strategy sub problems are non overlapping
binary search
Quicksort
What about problems where sub problems “share” sub problems?
Divide and conquer is inefficient in solving the shared problems more than once

10. Exponentiation – not so good…
public int pow(int b, int e) { if(e == 0) return 1; else if(e%2 == 0) return pow(b,e/2) * pow(b,e/2); else return b*pow(b,e-1); }

11. Exponentiation

12. Enter Dynamic Programming
public int pow(int b, int e) { int[] solved = new int[e+1]; return powDP(b,e,solved); } public int powDP(int b, int e, int[] solved) if(e == 0) solved[e] = 1; else if(solved[e] == 0) if(e%2 == 0) solved[e] = powDP(b,e/2,solved) * powDP(b,e/2,solved); else solved[e] = b * powDP(b,e-1,solved); return solved[e];

13. Fibonacci Consider the recursive solution to the Fibonacci numbers:
public int fib(int n) {
if(n == 0 || n == 1)
return 1;
else
return fib(n-1) + fib(n-2); }

14. Fibonacci cont’ Example call to F(6) F(6) = 1 F(5) = 1 F(4) = 2

15. Dynamic Programming
What if we were able to store the sub problems so they could be reused?
Blue box  already calculated so no sub calls

16. Top Down - Memoization
One approach to using dynamic programming is known as memoization
It mimics the idea of “saving” already solved sub problems
You will need to think ahead as to how many sub problems you may need to solve

17. Fibonacci Memoization
public int fib(int n) { int[] subProbs = new int[n+1]; return fibHelper(subProbs, n); } public int fibHelper(int[] subs, int n) if(n == 0 || n == 1) subs[n] = 1; else if(subs[n] == 0) subs[n] = fibHelper (subs, n - 1) + fibHelper (subs, n - 2); return subs[n];

18. Fibonacci Run

19. Bottom Up - Dynamic Programming
In bottom up dynamic programming we build up the sub problems until we reach the problem at hand
One major advantage is the possibility to remove recursion (and sometimes the memory required for memorization)

20. Fibonacci Bottom Up
public static int fib(int n) { int[] solns = new int[n+1]; solns[0] = 1; solns[1] = 1; for(int k = 2; k <= n; k++) solns[k] = solns[k-1] + solns[k-2]; return solns[n]; }

21. Fibonacci - Final
public static int fib(int n) { int f0 = 1; int fn = 1; for(int k = 2; k <= n; k++) fn = fn + f0; return fn; }

22. Dynamic Programming
Like Divide and Conquer and Recursive thinking, you need to practice
The comfortable you are with DP, the more likely you are to recognize it when solving a problem
The big hint here is that sub problems overlap AND contribute to the optimal solution of the given problem
F(5) requires F(4) and F(3) which overlap

23. Pascal’s Triangle
The first 7 rows:

24. Pascal’s Triangle
Recursive definition:

25. Programmer’s View Recursive definition: P(n,n) = 1 P(n,0) = 11 2 3 4 5 6 10 15 20
Recursive definition:
P(n,n) = 1
P(n,0) = 1
P(n,m) = P(n-1,m) + P(n-1,m-1)

26. Recursion
public int pascal(int r, int c) { if(r == c || c == 0) return 1; else return pascal(r – 1, c) + pascal(r – 1, c – 1); }

27. We’re better than that…
A simple loop is possible but… let’s look at how we can refine the problem using dynamic programming
P(4,2)  P(3, 2) + P(3,1)
P(3,2)  P(2,2) + P(2,1)
P(3,1)  P(2,1) + P(2,0)
P(2,2)  1
P(2,1)  P(1,1) + P(1,0)
P(2,0)  1
P(1,1)  1
P(1,0)  1

28. Dynamic Programming
public int pascal(int r, int c, int[][] solved) { if(r == c || c == 0) solved[r][c] = 1; else if(solved[r][c] == 0) solved[r][c] = pascal(r-1,c,solved) + pascal(r-1,c-1,solved); } return solved[r][c]; *Note: a 2D array is expensive! There are better ways if you know more math

29. Longest Increasing Subsequence
{7,3,4,2,1,9,6,8}
How would you program a computer to find the length of the longest increasing subsequence?
{3,4,6,8} = 4

30. Brute Force The LIS can be taken from any starting point in the array
Write a method, LIS(n) to find the LIS from a given starting point, n
Write a loop to look at every possible LIS(n) and return the largest

31. LIS(n)
public int incrSub(int[] list, int n) { if(n == list.length - 1) return 1; else int max = 1;//LIS of a non-empty list is 1 for(int i = n+1; i < list.length; i++)//search all possible elements after given one if(list[n] < list[i])//can join this element to subsequence int temp = 1 + incrSub(list, i); if(temp > max) max = temp; } return max;

32. public int lis(int[] list) { int longest = 1; for(int n = 0; n < list.length; n++) int temp = incrSub(list, n); if(temp > longest) longest = temp; } return longest; public int incrSub(int[] list, int n) if(n == list.length - 1) return 1; else int max = 1;//LIS of a non-empty list is 1 for(int i = n+1; i < list.length; i++)//search all possible elements after given one if(list[n] < list[i])//can join this element to subsequence int temp = 1 + incrSub(list, i); if(temp > max) max = temp; return max;
LIS

33. LIS - revised Brute force = O(n^3) 
Can you find where dynamic programming would apply here?
{7,3,4,2,1,9,6,8}
LIS(0) calls LIS(5)
LIS(1) calls LIS(2), LIS(5), LIS(6), LIS(7)
Let’s save each of these calls as we find them

34. public static int lisDP(int[] list) { int longest = 1; int[] solutions = new int[list.length]; for(int n = 0; n < list.length; n++) { int temp = incrSubDP(list, n, solutions); if(temp > longest) longest = temp; } return longest; public static int incrSubDP(int[] list, int n, int[] longest) { if(n == list.length - 1) longest[n] = 1; else if(longest[n] == 0) int max = 1;//LIS of a non-empty list is 1 for(int i = n+1; i < list.length; i++)//search all possible elements after given one if(list[n] < list[i])//can join this element to subsequence longest[i] = incrSubDP(list, i, longest); int temp = 1 + longest[i]; if(temp > max) max = temp; longest[n] = max; return longest[n];

35. Exercises This unit is a little bit different from the rest…
You will choose your own problems to solve
Choose 3 problems from the internet that use dynamic programming
Choose 2 CCC contest problems that use dynamic programming
In this case I’m not necessarily looking to see an original solution but for you to demonstrate your understanding of how to solve it using dynamic programming.
Learn from the solutions described on the internet, do not use the corresponding code
If you understand the problem and solution you should be able to program your own code
To do this you will teach me how to apply dynamic programming to solve your chosen problems.
I should see sample problems illustrating how the problem works (simple hand traced ones)
You should explain it to me so I can understand how you used dynamic programming

36. Contest Practice Complete 5 additional CCC problems online
They must be from S3 or higher
Register online
JH =