Greedy Algorithms CSE 331 Section 2 James Daly

Reminders Exam 2 next week Thursday, April 9th Covers heaps to spanning trees Greedy algorithms (today) and dynamic programming (Thursday) will not be on Exam 2 Homework 5 is out Not due until April 16 th (after the exam) Covers some testable material

Greedy Algorithms General algorithm design technique Used to solve optimization problems Multiple options exist, need to find the best one Ex. Shortest path Ex. Minimum spanning tree UV

Greedy Algorithms Make the choice that looks the best right now Make locally optimal choices at each step Expectation of globally good solution Does not always yield an optimal solution Usually simple and fast Ex. Prim’s and Kruskal’s algorithms

Key Ingredients Greedy Choice Property A global optimal solution can be achieved by making locally optimal (greedy) choices Optimal Substructure An optimal solution contains within it optimal solutions to subproblems

Example: Job Scheduling n jobs: j 1, j 2, j 3, … j n Time for executing job j i is t i We have one machine

Job Scheduling Problem Problem 1: What is the order of jobs that minimizes the total running time? Not an optimization problem Always the same cost T 1 = 4 T 2 = 3T 3 = 5T 4 = 6 T 1 = 4T 2 = 3T 3 = 5T 4 = 6

Job Scheduling Problem Problem 2: What is the order of jobs that minimizes the average job completion time Solution: Run the quickest job first ji “Opt” New ij xyz “Opt”: x + y + z New: x-w + y-kw + z (x + y + z) – (k+1) w < Opt w = T j - T i x-w y-kw z k items

Problem: Huffman Codes We want to send a long text document using as few bits as possible How can we encode the file to save space? File Compressed Received File Encode DecodeTransmit

Huffman Codes

What if we know the frequency of each symbol? a: 45%, b: 13%, c: 12%, d: 16%, e: 9%, f: 5% Let’s try a variable length code a: 0 b:101 c: 100 d:111 e: 1101 f: * (0.45*1 + ( ) * 3 + ( ) * 4) = 224,000 bits a a ac a bed f e

Prefix Codes No code word is a prefix of another code word Ex: a: 0 b: 01 c: 011 Ex: a: 0 b: 10 c: 11 Not allowed a c b

Optimization Problem

Huffman Coding Plan: Build a tree from all characters Combine least common characters into a new symbol

Example c 12 % e 9 % f 5 % a 45 % 100%55% b 13 % d 16 % 29%26%14% a 45 % b 13 % d 16 % c 12 %

Huffman(C) Q <- C // Build a heap While (Q.Size > 1): Left <- Q.ExtractMin() Right <- Q.ExtractMin() Freq <- Left.Freq + Right.Freq Node <- (Left, Right, Freq) Q.Insert(Node) Return Q.ExtractMin() Loop happens O(n) times Extraction takes O(log n) time As does insertion O(n log n) time total