1. 0-1 Knapsack Problem
A burglar breaks into a museum and finds “n” items
Let v_i denote the value of ith item, and let w_i denote the weight of the ith item
The burglar carries a knapsack capable of holding a total weight W
The burglar wishes to carry away the most valuable subset of items subject to the weight constraint
Want to steal diamonds before gold
We assume that the burglar cannot take a fraction of an object, so he/she must make a decision to take the object entirely or leave it behind
If the burglar can take a fraction of an object for a fraction of the value and weight, the problem very easily solved

2. 0-1 Knapsack: Formal Definition
Given <v1, v2, …, vn> and <w1, w2, …, wn>, and W > 0, we wish to determine the subset T <= {1, 2, .., n} (of objects to “take”) that maximizes
Sum_{i e T} vi
subject to the constraint
Sum_{i e T} wi <= W

3. 0-1 Knapsack: Brute-Force Algo
Brute-force algorithm would be to try all possible combinations of items that fit in the knapsack and see which one results in the highest total gain
How many possible sets?
Size of the superset of “n” items
Set of items containing 1 items: n
Set of items containing 2 items (n 2)
Set of items containing 3 items (n 3) …
Set of items containing n items (n n)
Sum_{k = 1}^n (n k) =? Exponential # of alternatives

4. 0-1 Knapsack: Greedy Algorithm
The greedy thing to do will be to compute the gain per unit of weight (vi/wi) for each item “i”, and fill in the knapsack starting with the item having the highest gain and continuing this way until the knapsack is filled.
Example next…

5. 0-1 Knapsack: Greedy Algorithm60 60 35 -- 40 60 60 40 30 40
$140
$160
$90 30 20 20 20 20
$100
$100
$100 10 5 5
$30 5
$30
-----
-----
-----
Knapsack
$30
$20
$100
$90
$160
$270
$220
$260
Gain
Greedy solution to
Fractional problem
Greedy solution to
0-1 problem
Optimal solution to
0-1 problem

6. 0-1 Knapsack: Greedy Algorithm
Greedy Algorithm does not work because we cannot take fraction of an item. If that was allowed, the greedy algorithm would have found the optimum solution
How do we solve 0-1 knapsack then?
It turns out this problem is NP-complete, so we cannot hope to find an efficient algorithm
However, if we make the same sort of assumptions that we made in counting sort, we can come up with an efficient algorithm

7. Formulation of the Problem
We have object {1, 2, 3, …, n}., Knapsack of size: W
Here is how we formulate the problem:
Assume you have a function Knapsack, that that computes the optimal Value: Knapsack(v, w, n, W).
Leave the Last Object:
If we choose to not take object “n”, then the optimal value will come about by considering how to fill a knapsack of size W with the remaining objects {1, 2, …, i-1}. This is Knapsack(v, w, n-1, W)
Take the Last Object:
If we take object “n”, then we gain a value of v[n], but have used up w[n] of our capacity. With remaining W-w[n] capacity in the knapsack, we can fill it up to the best possible way with objects {1, 2, …, i-1}. This is v[n]+Knapsack(v, w, n-1, W-w[n]). This is only possible if w[n] <= W

8. 0-1 Knapsack: Divide-Conquer Algorithm
Knapsack(v[1..n], w[1..n], n, W){
if (n <= 0) return 0;
if (W <= 0) return 0;
// Consider leaving object “n”
leave_val = Knapsack(v, w, n-1, W);
// Consider taking object “n”
take_val = -INFINITY;
if (w[n] <= W) take_val = v[n] + Knapsack(v, w, n-1, W-w[n]);
return max(leave_val, take_val);
} //end-Knapsack
Running Time? O(2n) in the worst case
Notice that the function makes 2 calls to itself.
At each call, n is decremented by 1. So the height of the function call tree is potentially “n”, giving rise to potentially 2n function invocations

9. 0-1 Knapsack: DP Solution
The problem with the divide and conquer algorithm is that it solves the same subproblems over and over again.
This is what DP is all about. To make divide and conquer algorithms more efficient
The idea is to store the solutions to subproblems in a table and retrieve the solutions from that table the next time we need to solve the same subproblem. This way each subproblem is solved just once, giving rise to efficient algorithms

10. 0-1 Knapsack: DP Solution
How to store solutions to subproblems?
Construct an array V[0..n, 0..W]
For 1 <= i <= n, 0 <= j <= W, the entry V[i, j] will store the maximum value of any subset of objects {1, 2, .., i} that can fit into a knapsack of weight “j”
If we can compute all the entries of this array, then the entry V[n, W] will contain the max. value of all “n” objects that can fit into the entire knapsack of weight W

11. Computing V[i, j] Observe that V[0, j] = 0 for all 0 <= j <= W
No items, no value
We consider 2 cases:
Leave Object:
If we choose to not take object “i”, then the optimal value will come about by considering how to fill a knapsack of size “j” with the remaining objects {1, 2, …, i-1}. This is just V[i-1, j]
Take Object:
If we take object “i”, then we gain a value of vi, but have used up wi of our capacity. With remaining j-wi capacity in the knapsack, we can fill it up to the best possible way with objects {1, 2, …, i-1}. This is vi+V[i-1, j-wi]. This is only possible if wi <= j

12. 0-1 Knapsack: Final Formulation
Combining all observations, we have the following rules
{ if i=0 and 0<=j<=W
v[i,j] ={v[i-1,j] if wi > j
{max{v[i-1,j], vi+v[i-1,j-wi] if wi<=j

13. 0-1 Knapsack: Algorithm Knapsack(v[1..n], w[1..n], n, W){
Allocate V[0..n][0..W];
For j=0 to W do V[0, j] = 0; // Initialization
For i=1 to n do {
For j=0 to W do {
leave_val = V[i-1, j]; // Total val. If we leave i
if (j >= w[i]) // Enough capacity to take i?
take_val = v[i] + V[i-1, j-w[i]]; // Tot. value if
else // we take i
take_val = -INFINITY; // cannot take i
V[i, j] = max{leave_val, take_val}; // final value
} //end-for
return V[n, W];
} //end-Knapsack

14. 0-1 Knapsack: Running Time
It is easy to see that the algorithm takes (n+1)*(W+1) = O(n*W) steps
The algorithm computes the maximum attainable knapsack value in V[n, W], but does not describe which items are taken
But that can be added by recording for each entry V[i, j] in the matrix whether we got this entry by taking the ith item, or leaving it.
This this information it is possible to reconstruct the optimum knapsack contents

15. 0-1 Knapsack: Example Values of the objects are <10, 40, 30, 50>
Weights of the objects are <5, 4, 6, 3>
The capacity of the knapsack, W = 10
Capacity
J = 0 1 2 3 4 5 6 7 8 9 10
Item
Value
Weight 1 10 5 10 10 10 10 10 10 2 40 4 40 40 40 40 40 50 50 3 30 6 40 40 40 40 40 50 70 4 50 3 50 50 50 50 90 90 90 90
Final result is V[4, 10] = 90 (for taking items 2 and 4)