1. Lecture 9 Greedy Algorithms

2. Example 3: Horn-SAT Logical formulas Variable 𝑥 – true or false
Negation 𝑥
Horn Clauses 𝑥∧𝑦⇒𝑧 𝑥 𝑥 ∨ 𝑦 ∨ 𝑧
Problem: Given a set of Horn clauses, determine whether there exists an assignment to variables such that all clauses are satisfied.

3. Different Clauses
𝑥 1 ∧ 𝑥 4 ⇒ 𝑥 2 𝑥 1 ∧ 𝑥 2 ⇒ 𝑥 3 𝑥 1 𝑥 𝑥 1 ∨ 𝑥 𝑥 1 ∨ 𝑥 2 ∨ 𝑥 4
Type 1: If some variables are true, x is true
Type 2: Variable x is true
Type 3: At least one of these variables is false.
Algorithm:
First set everything to false
x1 = true (clause 3); x2 = true (clause 4) x3 = true (clause 2); check clauses 5 and 6, clause 5 is violated, output “No”

4. Proof Idea
If the algorithm succeeds, then there is clearly an assignment
Need to show: if the algorithm fails, then no assignment can satisfy the clauses
Assume there is actually an assignment, how to find a contradiction?

5. Proof Sketch
𝑥 1 ∧ 𝑥 4 ⇒ 𝑥 2 𝑥 1 ∧ 𝑥 2 ⇒ 𝑥 3 𝑥 1 𝑥 𝑥 1 ∨ 𝑥 𝑥 1 ∨ 𝑥 2 ∨ 𝑥 4
Algorithm: x1=x2=x3 = true x4=false
Case 1: If the satisfying assignment also has x1=x2=x3 = true
Then clause 5 is violated
So one of x1,x2,x3 must be false, look for the first variable that is false.

6. Proof Sketch
𝑥 1 ∧ 𝑥 4 ⇒ 𝑥 2 𝑥 1 ∧ 𝑥 2 ⇒ 𝑥 3 𝑥 1 𝑥 𝑥 1 ∨ 𝑥 𝑥 1 ∨ 𝑥 2 ∨ 𝑥 4
Algorithm: x1=x2=x3 = true x4=false
Case 2: If x1 or x2 if false
Then clause 3 or 4 is violated

7. Proof Sketch
𝑥 1 ∧ 𝑥 4 ⇒ 𝑥 2 𝑥 1 ∧ 𝑥 2 ⇒ 𝑥 3 𝑥 1 𝑥 𝑥 1 ∨ 𝑥 𝑥 1 ∨ 𝑥 2 ∨ 𝑥 4
Algorithm: x1=x2=x3 = true x4=false
Case 3: If x3 is the first variable that is false
Both x1 and x2 are true.
Clause 2 is violated.

8. Example 4: The Encoding Problem (Hoffman Tree)
Problem: Given a long string with n different characters in alphabet, find a way to encode these characters into binary codes that minimizes the length.
Example: “aababc”, n = 3, alphabet = {a,b,c}
If a = ‘0’, b = ’11’, c = ‘10’, then the string can be encoded as ‘ ’.

9. Why don’t just use ASCII?
Different letters appear with very different frequency.
Using an encoding of varying length can save space.
Now of course memory/disk are cheap, but some communications can be expensive.

10. What kind of encodings are allowed?
Bad example: If a = ‘0’, b = ’01’, c = ‘1’
Given encoding , it can be ‘aababc’, but can also be ‘aaacabc’ (and others)
Want an encoding that allow unique decoding.
Sufficient condition: Prefix free encoding
Definition: No code should be a prefix of another code.
(In previous example, encoding of a is a prefix for encoding of b.)

11. Prefix-free encoding vs. Trees1
a = 00
b = 01
c = 110
d = 111 1 1 a b 1 c d
Character = Leaves of the Tree
Encoding = Path from root to the character
Prefix free = no characters for intermediate nodes

12. Cost of the Tree Cost = sum of Frequency * Depth1
Cost = sum of Frequency * Depth
= 5*2 + 10*2 + 3*3 + 6*3
= 57 1 1 a b 1 5 10 c d 3 6

13. Hoffman Tree problem
Given a long string with n different characters in alphabet, find a way to encode these characters into binary codes that minimizes the length.
Given an alphabet of n characters, character i appears wi times, find an encoding tree with minimum cost.

14. Cost of the Tree - Revisited1
Cost = 9 = 3 + 6 1 1 a b c d 5 10 3 6
Claim: Cost of the tree = sum of cost for each node.

15. Algorithm Hoffman Tree REPEAT
Select two characters with smallest frequencies
Merge them into a new character, whose frequency is the sum
UNTIL there is only one character
abcd 24
acd 14 ac 8 a 5 b 10 c 3 d 6

16. Proof Sketch
Use induction: Assume algorithm finds the optimal solution for alphabet of size n, we want to prove that it works for alphabet of size n+1.
Base Case: n = 1 is trivial. (cost = 0 in this case)

17. Induction Step Goal: Convince OPT that it is OK to merge i, j first. i
First merged i, j (characters with lowest frequency)
i, j may be in different branches
Goal: Convince OPT that it is OK to merge i, j first.

18. Induction Step i j j i
After ALG and OPT both merged i, j, it reduces to a problem of size n!
Induction hypothesis ALG is optimal, OPT cannot be better.