UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2006 Design Patterns for Optimization Problems Dynamic Programming & Greedy Algorithms

Algorithmic Paradigm Context Subproblem solution order Make choice, then solve subproblem(s) Solve subproblem(s), then make choice

Activity Selection Optimization Problem ä Problem Instance: ä Set S = {1,2,...,n} of n activities ä Each activity i has: ä start time: s i ä finish time: f i ä Activities i, j are compatible iff non-overlapping: ä Objective: ä select a maximum-sized set of mutually compatible activities source: textbook Cormen, et al.

Activity Selection Algorithmic Progression ä “Brute-Force” ä (board work) ä Dynamic Programming #1 ä Exponential number of subproblems ä (board work) ä Dynamic Programming #2 ä Quadratic number of subproblems ä (board work) ä Greedy Algorithm ä (board work)

Dynamic Programming Activity Selection

Dynamic Programming Approach to Optimization Problems 1. Characterize structure of an optimal solution. 2. Recursively define value of an optimal solution. 3. Compute value of an optimal solution in bottom-up fashion. 4. Construct an optimal solution from computed information. source: textbook Cormen, et al.

Activity Selection Solution to S ij including a k produces 2 subproblems: 1) S ik (start after a i finishes; finish before a k starts) 2) S kj (start after a k finishes; finish before a j starts) source: textbook Cormen, et al. c[i,j]=size of maximum-size subset of mutually compatible activities in S ij.

Greedy Algorithms

What is a Greedy Algorithm? ä Solves an optimization problem ä Optimal Substructure: ä optimal solution contains in it optimal solutions to subproblems ä Greedy Strategy: ä At each decision point, do what looks best “locally” ä Choice does not depend on evaluating potential future choices or presolving overlapping subproblems ä Top-down algorithmic structure ä With each step, reduce problem to a smaller problem ä Greedy Choice Property: ä “locally best” = globally best

Greedy Strategy Approach 1. Determine the optimal substructure of the problem. 2. Develop a recursive solution. 3. Prove that, at any stage of the recursion, one of the optimal choices is the greedy choice. 4. Show that all but one of the subproblems caused by making the greedy choice are empty. 5. Develop a recursive greedy algorithm. 6. Convert it to an iterative algorithm. source: textbook Cormen, et al.

source: web site accompanying textbook Cormen, et al. Errors from earlier printing are corrected in red. High-level call: RECURSIVE-ACTIVITY-SELECTOR(s,f,0,n) Returns an optimal solution for Recursive Greedy Activity Selection

source: web site accompanying textbook Cormen, et al.

Running time? Iterative Greedy Activity Selection ä Iterative Greedy Algorithm: ä S’ = presort activities in S by nondecreasing finish time ä and renumber ä GREEDY-ACTIVITY-SELECTOR(S’) ä n length[S’] ä A {1} ä j1 ä for i 2 to n ä do if ä then ä j i ä return A source: textbook Cormen, et al.

Streamlined Greedy Strategy Approach 1. View optimization problem as one in which making choice leaves one subproblem to solve. 2. Prove there always exists an optimal solution that makes the greedy choice. 3. Show that greedy choice + optimal solution to subproblem optimal solution to problem. source: textbook Cormen, et al. Greedy Choice Property: “locally best” = globally best