1. Greedy Algorithms
Analysis of Algorithms

2. Greedy Algorithm Paradigm
Characteristics of greedy algorithms:
make a sequence of choices
each choice is the one that seems best so far, only depends on what's been done so far
choice produces a smaller problem to be solved

3. Designing a Greedy Algorithm
Cast the problem so that we make a greedy (locally optimal) choice and are left with one subproblem
Prove there is always a (globally) optimal solution to the original problem that makes the greedy choice
Show that the choice together with an optimal solution to the subproblem gives an optimal solution to the original problem

4. Some Greedy Algorithms
Coin changing algorithm
Huffman codes
Kruskal's MST algorithm
Prim's MST algorithm
Dijkstra's Shortest Path algorithm …

5. Making Change Input Task Example Positive integer n
Compute a minimal multiset of coins from C = {d1, d2, d3, …, dk} such that the sum of all coins chosen equals n
Example
n = 73, C = {1, 5, 10, 25}
Solution: 3 coins of size 20, 1 coin of size 10, 3 coins of size 1

6. Greedy Algorithm Suppose we have n types of coins with values
d1 > d2 > … > dn > 0
Input: A positive integer C
m[1] := 0; m[2]:=0; … ; m[n] := 0;
for i = 1 to n do
while C > di do
C := C – di; m[i]++;
end while;
end for;

7. Knapsack Problem There are n different items in a store Item i :
weighs wi pounds
worth $vi
A thief breaks in
Can carry up to W pounds in his knapsack
What should he take to maximize the value of his haul?

8. Greedy Algorithm 3 items: Knapsack can hold 50 lbs Greedy algorithm:
item 1 weighs 10 lbs, worth $60 ($6/lb)
item 2 weighs 20 lbs, worth $100 ($5/lb)
item 3 weighs 30 lbs, worth $120 ($4/lb)
Knapsack can hold 50 lbs
Greedy algorithm:
take item 1
take item 2
no room for item 3

9. Finding Optimal Code Input: Output: data file of characters and
number of occurrences of each character
Output:
a binary encoding of each character so that the data file can be represented as efficiently as possible
"optimal code"

10. Huffman Code
Idea: use short codes for more frequent characters and long codes for less frequent
char a b c d e f
total bits # 45 13 12 16 9 5
fixed
000
001
010
011
100
101
300
variable
111
1101
1100
224

11. How to Decode? 101111110100 With fixed length code, easy:
break up into 3's, for instance
For variable length code, ensure that no character's code is the prefix of another
no ambiguity
b d e a a

12. Tree Representation fixed length code 1 cost of code is sum,1
cost of code is sum,
over all chars c, of
number of occurrences
of c times depth of c in
the tree 1 1 1 1 a b c d e f

13. Tree Representation 1 variable length code a 1 cost of code is sum,1
variable length code a 1
cost of code is sum,
over all chars c, of
number of occurrences
of c times depth of c in
the tree 1 1 b c 1 d f e

14. Greedy Algorithm Given set C of n chars, c occurs f[c] times
Insert each c into priority queue Q using f[c] as key
for i := 1 to n-1 do
x := extract-min(Q);
y := extract-min(Q);
make a new node z w/ left child x (label edge 0), right child y (label edge 1), and f[z] = f[x] + f[y];
insert z into Q;

15. Matroid
Let S be a finite set, and F a nonempty family of subsets of S, that is, F P(S).
We call (S,F) a matroid if and only if
M1) If BF and A  B, then AF [hereditary]
M2) If A,BF and |A|<|B|, then there exists x in B\A such that A{x} in F [exchange property]
Matroid is useful when determining whether greedy technique yields optimal solutions.
It covers many cases of practical interests.