Greedy Algorithms Analysis of Algorithms

Greedy Algorithm Paradigm Characteristics of greedy algorithms: make a sequence of choices each choice is the one that seems best so far, only depends on what's been done so far choice produces a smaller problem to be solved

Designing a Greedy Algorithm Cast the problem so that we make a greedy (locally optimal) choice and are left with one subproblem Prove there is always a (globally) optimal solution to the original problem that makes the greedy choice Show that the choice together with an optimal solution to the subproblem gives an optimal solution to the original problem

Some Greedy Algorithms Coin changing algorithm Huffman codes Kruskal's MST algorithm Prim's MST algorithm Dijkstra's Shortest Path algorithm …

Making Change Input Task Example Positive integer n Compute a minimal multiset of coins from C = {d1, d2, d3, …, dk} such that the sum of all coins chosen equals n Example n = 73, C = {1, 5, 10, 25} Solution: 3 coins of size 20, 1 coin of size 10, 3 coins of size 1

Greedy Algorithm Suppose we have n types of coins with values d1 > d2 > … > dn > 0 Input: A positive integer C m[1] := 0; m[2]:=0; … ; m[n] := 0; for i = 1 to n do while C > di do C := C – di; m[i]++; end while; end for;

Knapsack Problem There are n different items in a store Item i : weighs wi pounds worth $vi A thief breaks in Can carry up to W pounds in his knapsack What should he take to maximize the value of his haul?

Greedy Algorithm 3 items: Knapsack can hold 50 lbs Greedy algorithm: item 1 weighs 10 lbs, worth $60 ($6/lb) item 2 weighs 20 lbs, worth $100 ($5/lb) item 3 weighs 30 lbs, worth $120 ($4/lb) Knapsack can hold 50 lbs Greedy algorithm: take item 1 take item 2 no room for item 3

Finding Optimal Code Input: Output: data file of characters and number of occurrences of each character Output: a binary encoding of each character so that the data file can be represented as efficiently as possible "optimal code"

Huffman Code Idea: use short codes for more frequent characters and long codes for less frequent char a b c d e f total bits # 45 13 12 16 9 5 fixed 000 001 010 011 100 101 300 variable 111 1101 1100 224

How to Decode? 101111110100 With fixed length code, easy: break up into 3's, for instance For variable length code, ensure that no character's code is the prefix of another no ambiguity 101111110100 b d e a a

Tree Representation fixed length code 1 cost of code is sum, 1 cost of code is sum, over all chars c, of number of occurrences of c times depth of c in the tree 1 1 1 1 a b c d e f

Tree Representation 1 variable length code a 1 cost of code is sum, 1 variable length code a 1 cost of code is sum, over all chars c, of number of occurrences of c times depth of c in the tree 1 1 b c 1 d f e

Greedy Algorithm Given set C of n chars, c occurs f[c] times Insert each c into priority queue Q using f[c] as key for i := 1 to n-1 do x := extract-min(Q); y := extract-min(Q); make a new node z w/ left child x (label edge 0), right child y (label edge 1), and f[z] = f[x] + f[y]; insert z into Q;

Matroid Let S be a finite set, and F a nonempty family of subsets of S, that is, F P(S). We call (S,F) a matroid if and only if M1) If BF and A  B, then AF [hereditary] M2) If A,BF and |A|<|B|, then there exists x in B\A such that A{x} in F [exchange property] Matroid is useful when determining whether greedy technique yields optimal solutions. It covers many cases of practical interests.