Algorithms CSCI 235, Spring 2019 Lecture 30 More Greedy Algorithms

The Knapsack problem The problem: A thief robbing a store finds n items. The ith item weighs wi pounds and has worth vi. The thief can only carry W pounds in his knapsack. Which items should he take (to get the maximum value)? Version 1 (the 0-1 version): The thief can only take each item whole. He cannot take fractional items (e.g. gold bars). Version 2 (the fractional version): The thief can take fractions of items (e.g. gold dust). Applications: There are many real-world problems analagous to the Knapsack problem. Example: The bin-packing problem. Objects of different sizes must be packed into bins of volume V in a way that minimizes the number of bins used. This problem occurs for loading trucks and for creating file back-ups in media.

Optimal Substructure of fractional problem The fractional version: Suppose the thief takes w pounds of item j first. There are wj - w pounds of item j left. The thief can fit W-w pounds in his knapsack. We now must find the most valuable load weighing <= W-w from the remaining items and wj - w pounds of the jth item.

Solving the fractional problem We can solve the fractional problem using a greedy algorithm: 1) Compute the value per pound of each item: vi/wi 2) The thief takes as much as possible of the most valuable (per pound) item. 3) If the thief takes all of the most valuable item and still has room in the knapsack, he takes as much as possible of the next most valuable item. 4) Repeat step 3 until the knapsack is full. Suppose the knapsack can hold 50 lbs. Suppose there are 3 items: Item Weight Value Price/lb 1 10 lbs $60 $6 2 20 lbs $100 $5 3 30 lbs $120 $4 The fractional solution: (We will work through this in class)

Optimal Substructure of 0-1 problem The 0-1 version: Suppose the thief takes item j first. Item j weighs wj pounds. Now the thief can carry W - wj pounds. Now we must solve the subproblem of finding the most valuable combination of the remaining items weighing <= W-wj. We must find an optimal solution to this subproblem. Applying the same strategy to the 0-1 problem does not lead to an optimal solution. To find the optimal solution to the 0-1 problem we must compare solutions in which the first item is included with solutions in which it is not included. The problems overlap. We must solve the subproblems to make the choice. This means we must use dynamic programming to find the optimal solution.

Huffman Codes Huffman codes are a way to compress data. They are widely used and effective, leading to 20 - 90% savings of space, depending on the data. A greedy algorithm uses the frequency and occurrence of each character to build an optimal code for representing each character as a binary string.

Example Suppose we have 105 characters in a data file. Normal storage: 8 bits per character (ASCII) -- 8x105 bits in file. We want to compress the file and store it compactly. Suppose only 6 characters appear in the file: a b c d e f Total freq 45 13 12 16 9 5 100 How can we represent the data in a compact way? Fixed Length code: Each letter represented by an equal number of bits. (We will give an example in class).

Variable length codes We can save space by assigning frequently occurring characters short codes, and infrequently occurring characters long codes. a b c d e f Total freq 45 13 12 16 9 5 100 Example: a 0 b 101 c 100 d 111 e 1101 f 1100 Number of bits = ?

Prefix Codes Prefix codes are codes in which no codeword is a prefix for another code word. We can show that optimal data compression achievable by a character code can be achieved by a prefix code. Prefix codes simplify encoding and decoding. To encode: concatenate the code for each character: abc = 0101100 To decode: There is no ambiguity, assign first character that fits. 001011101= ?

Representing codes with a tree A tree is a convenient representation for prefix codes so codewords can be easily decoded. Find character by traversing tree from root to leaf. Move to left if 0. Move to right if 1. Code 1 a 000 b 001 c 010 d 011 e 100 f 101 We will draw the tree in class Leaf value = frequency of character. Node value = sum of values of children = sum of leaves in subtrees.

Tree for variable length code We will draw this tree in class Code 2 a 0 b 101 c 100 d 111 e 1101 f 1100 Only full binary trees (every non-leaf node has exactly 2 children) represent optimal codes. Code 2 is optimal. Code 1 is not.

Size of Tree The tree for an optimal prefix code has 1 leaf for each letter in the alphabet. Let C = alphabet from which the set of characters is drawn. |C| = Size of C (number of characters in the alphabet) Then the number of leaves in the tree = |C| and the number of internal nodes in the tree = |C| -1 The number of bits to encode a given character is given by the depth (dT(c)) of the leaf that contains that character. The depth is the length of the path from root to leaf. Given a tree, T, corresponding to a prefix code, Number of bits to encode a file is: This is the cost of the tree. frequency of c depth of c