Dynamic Programming (DP) Simpler, Better, Faster Carl Hultquist

Rough definition Dynamic programming is: Dynamic programming is not: Breaking a problem up into smaller sub-problems By finding the optimal solution to these smaller sub-problems, being able to find the solution to the bigger problem Dynamic programming is not: A type of programming language (like declarative and imperative)

Learn by example: Fibonacci numbers We all know the Fibonacci numbers 1 1 2 3 5 8 13… and that we can write a neat definition for these as: f(0) = 1 f(1) = 1 f(n) = f(n – 2) + f(n – 1) for n>1

Fibonacci in code int f(int n) { if (n < 2) return 1; else return f(n – 2) + f(n – 1); }

Fibonacci and big-O So we have an algorithm for finding f(n), but is it any good? What’s the big-O for finding f(n)? Answer: O(f(n)) So… is this good? Is it bad?

Recursive Fibonacci: bad As an indication, f(1000)=1,318,412,525 Surely we can do better…

The trick: overlapping subproblems For some n, f(n)=f(n-2)+f(n-1) Now f(n-1)=f(n-3)+f(n-2) Note the common f(n-2) – so to calculate the value of f(n), we actually calculate f(n-2) twice. Doing this using our recursive program is wasteful – we should only need to work it out once!!!

A better Fibonacci #define MAX_N 10000 int f[MAX_N]; int fib(int n) { f[0] = f[1] = 1; for (int i = 2; i <= n; i++) f[i] = f[i – 2] + f[i – 1]; return f[n]; }

Coin counting  Yes, today’s problem was a DP. So we want to make change of value M, we have N coin denominations, and their values are Vi for i=1…N. Now we can write the solution like this: coins(M) = min{coins(M – V1), coins(M – V2), …, coins(M – VN)} + 1 Now, doesn’t that look just a little bit like the Fibonacci definition? ;-)

A better solution… int N, M; int V[N]; int coins[M + 1]; coins[0] = 0; for (int i = 1; i <= M; i++) { int best = M; for (int j = 0; j < N; j++) if (V[j] <= i && coins[i – V[j]] + 1 < best) best = coins[i – V[j]] + 1; coins[i] = best; }

Checking for valid states int N, M; int V[N]; int coins[M + 1]; set(coins[0], coins[M], -1); coins[0] = 0; for (int i = 1; i <= M; i++) { int best = M; for (int j = 0; j < N; j++) if (V[j] <= i && coins[i – V[j]] != -1 && coins[i – V[j]] + 1 < best) best = coins[i – V[j]] + 1; coins[i] = best; }

An alternative coin solution: memoization int N, M; int V[N]; int cache[M + 1]; set(cache[0], cache[M], -1); int coins(int amount) { if (cache[amount] == -1) int best = M; for (int i = 0; i < N; i++) if (V[i] <= amount && coins(amount) + 1 < best) best = coins(amount) + 1; cache[amount] = best; } return cache[amount];

Two approaches to DP Top-down: recursion + memoization. Easier to code, but may have greater memory requirements. Also has extra stack + function call overhead. Bottom-up: iterative, solves subproblems from the smallest one up. Sometimes harder to code and/or work out exactly what’s going on, but more efficient.

Backtracking Some DP problems just call for the value of the best answer (like today’s problem). Others make your life harder: they ask for the “path” to the solution. Today’s problem could have done this by asking you to output the actual coins used

Backtracking (2) This usually isn’t too hard: in the same way that you have an array to store your best solution, you also keep an array indicating how you got there. For the coins problem, this array can simply store the last coin used to reach each total You can then “backtrack” by subtracting the coin value from the total, and look at the next coin that was needed. Repeat until you hit 0 

Coins with backtrack int N, M; int V[N]; int coins[M + 1]; int coinUsed[M + 1]; coins[0] = 0; for (int i = 1; i <= M; i++) { int best = M; int coin = -1; for (int j = 0; j < N; j++) if (V[j] <= i && coins[i – V[j]] + 1 < best) best = coins[i – V[j]] + 1; coin = j; } coins[i] = best; coinUsed[i] = coin;

Higher dimensions So far, we’ve just seen 1D DP problems At the IOI, 2D (or higher!) problems crop up often Consider this one: given a NxN grid of numbers and M co-ordinate pairs (a,b);(c,d), find the sum of the values in the grid for each of the rectangles with top-left co-ordinate (a,b) and bottom-right co-ordinate (c,d).

The naïve approach For each of the M rectangles, loop over the rectangle and sum up the values. This has O(N2M). With DP, we can instead get O(N2+M) which will usually be better (unless M is huge)

Getting clever… Suppose we can compute and store the sum of all rectangles with top-left co-ordinate (1,1). Let’s stash these in a 2D array called sum[][]. Then, to work out the sum of a rectangle from (a,b) to (c,d), we can work it out using our sum array like this: sum(a,b,c,d) = sum[c][d] – sum[c][b-1] – sum[a-1][d] + sum[a-1][b-1]

Cool, now how do we create our sum array? Suppose the grid values are in an array called grid[][]. To work out sum[i][j], we just need to see that it can be calculated as: sum[i][i] = sum[i][j-1] + sum[i-1][j] – sum[i-1][j-1] + grid[i][j]

… which we can do with a quick 2D loop sum[0][0] = 0; sum[0][j] = 0; sum[i][0] = 0; for (int i = 1; i <= N; i++) for (int j = 1; j <= N; j++) sum[i][j] = sum[i-1][j] + sum[i][j-1] - sum[i-1][j-1] + grid[i][j];

Common applications Longest common subsequence problem Floyd’s all-pairs shortest path Knapsack problem Duckworth-Lewis method! … amongst many others. See: http://en.wikipedia.org/wiki/Dynamic_Programming

Questions?