1 Algorithms CSCI 235, Fall 2015 Lecture 30 More Greedy Algorithms

2 The Knapsack problem The problem: A thief robbing a store finds n items. The i th item weighs w i pounds and has worth v i. The thief can only carry W pounds in his knapsack. Which items should he take (to get the maximum value)? Version 1 (the 0-1 version): The thief can only take each item whole. He cannot take fractional items (e.g. gold bars). Version 2 (the fractional version): The thief can take fractions of items (e.g. gold dust). Applications: There are many real-world problems analagous to the Knapsack problem. Example: The bin-packing problem. Objects of different sizes must be packed into bins of volume V in a way that minimizes the number of bins used. This problem occurs for loading trucks and for creating file back-ups in media.

3 Optimal Substructure of fractional problem The fractional version: Suppose the thief takes w pounds of item j first. There are w j - w pounds of item j left. The thief can fit W-w pounds in his knapsack. We now must find the most valuable load weighing <= W-w from the remaining items and w j - w pounds of the jth item.

Solving the fractional problem We can solve the fractional problem using a greedy algorithm: 1) Compute the value per pound of each item: vi/wi 2) The thief takes as much as possible of the most valuable (per pound) item. 3) If the thief takes all of the most valuable item and still has room in the knapsack, he takes as much as possible of the next most valuable item. 4) Repeat step 3 until the knapsack is full. Suppose the knapsack can hold 50 lbs. Suppose there are 3 items: ItemWeightValuePrice/lb 110 lbs$60$6 220 lbs$100$5 330 lbs$120$4 The fractional solution: (We will work through this in class)

5 Optimal Substructure of 0-1 problem The 0-1 version: Suppose the thief takes item j first. Item j weighs w j pounds. Now the thief can carry W - w j pounds. Now we must solve the subproblem of finding the most valuable combination of the remaining items weighing <= W-w j. We must find an optimal solution to this subproblem. Applying the same strategy to the 0-1 problem does not lead to an optimal solution. (We will show this in class).

6 The problem continued To find the optimal solution to the 0-1 problem we must compare solutions in which the first item is included with solutions in which it is not included. The problems overlap. We must solve the subproblems to make the choice. This means we must use dynamic programming to find the optimal solution.

7 Huffman Codes Huffman codes are a way to compress data. They are widely used and effective, leading to % savings of space, depending on the data. A greedy algorithm uses the frequency and occurrence of each character to build an optimal code for representing each character as a binary string.

8 Example Suppose we have 10 5 characters in a data file. Normal storage: 8 bits per character (ASCII) -- 8x10 5 bits in file. We want to compress the file and store it compactly. Suppose only 6 characters appear in the file: abcdefTotal freq How can we represent the data in a compact way? Fixed Length code: Each letter represented by an equal number of bits. (We will give an example in class).

9 Variable length codes We can save space by assigning frequently occurring characters short codes, and infrequently occurring characters long codes. abcdefTotal freq Example: a0a0 b101 c100 d111 e1101 f1100 Number of bits = ?

10 Prefix Codes Prefix codes are codes in which no codeword is a prefix for another code word. We can show that optimal data compression achievable by a character code can be achieved by a prefix code. Prefix codes simplify encoding and decoding. To encode: concatenate the code for each character: abc = To decode: There is no ambiguity, assign first character that fits = ?

11 Representing codes with a tree A tree is a convenient representation for prefix codes so codewords can be easily decoded. Code 1 a 000 b 001 c 010 d 011 e 100 f 101 Leaf value = frequency of character. Node value = sum of values of children = sum of leaves in subtrees. Find character by traversing tree from root to leaf. Move to left if 0. Move to right if 1. We will draw the tree in class

12 Tree for variable length code Code 2 a 0 b 101 c 100 d 111 e 1101 f 1100 We will draw this tree in class Only full binary trees (every non-leaf node has exactly 2 children) represent optimal codes. Code 2 is optimal. Code 1 is not.