1. UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2006 Lecture 2 Monday, 2/6/06 Design Patterns for Optimization Problems Greedy Algorithms

2. Algorithmic Paradigm Context Subproblem solution order Make choice, then solve subproblem(s) Solve subproblem(s), then make choice

3. Greedy Algorithms

4. What is a Greedy Algorithm? ä Solves an optimization problem ä Optimal Substructure: ä optimal solution contains in it optimal solutions to subproblems ä Greedy Strategy: ä At each decision point, do what looks best “locally” ä Choice does not depend on evaluating potential future choices or presolving overlapping subproblems ä Top-down algorithmic structure ä With each step, reduce problem to a smaller problem ä Greedy Choice Property: ä “locally best” = globally best

5. Greedy Strategy Approach 1. Determine the optimal substructure of the problem. 2. Develop a recursive solution. 3. Prove that, at any stage of the recursion, one of the optimal choices is the greedy choice. 4. Show that all but one of the subproblems caused by making the greedy choice are empty. 5. Develop a recursive greedy algorithm. 6. Convert it to an iterative algorithm. source: 91.503 textbook Cormen, et al.

6. Examples ä Activity Selection ä Minimum Spanning Tree ä Dijkstra Shortest Path ä Huffman Codes ä Fractional Knapsack

7. Activity Selection

8. Example Optimization Problem: Activity Selection ä Problem Instance: ä Set S = {1,2,...,n} of n activities ä Each activity i has: ä start time: s i ä finish time : f i ä Activities i, j are compatible iff non-overlapping: ä Objective: ä select a maximum-sized set of mutually compatible activities source: 91.503 textbook Cormen, et al.

10. Algorithmic Progression ä “Brute-Force” ä (board work) ä Dynamic Programming #1 ä Exponential number of subproblems ä (board work) ä Dynamic Programming #2 ä Quadratic number of subproblems ä Greedy Algorithm

11. Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. source: web site accompanying 91.503 textbook Cormen, et al. Errors from earlier printing are corrected in red. High-level call: RECURSIVE-ACTIVITY-SELECTOR(s,f,0,n)

12. Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. source: web site accompanying 91.503 textbook Cormen, et al.

13. Running time? Greedy Algorithm ä Algorithm: ä S’ = presort activities in S by nondecreasing finish time ä and renumber ä GREEDY-ACTIVITY-SELECTOR(S’) ä n length[S’] ä A {1} ä j1 ä for i 2 to n ä do if ä then ä j i ä return A source: 91.503 textbook Cormen, et al.

14. Streamlined Greedy Strategy Approach 1. View optimization problem as one in which making choice leaves one subproblem to solve. 2. Prove there always exists an optimal solution that makes the greedy choice. 3. Show that greedy choice + optimal solution to subproblem optimal solution to problem. source: 91.503 textbook Cormen, et al. Greedy Choice Property: “locally best” = globally best

15. Minimum Spanning Tree

16. A B C D E F G 2 2 1 1 3 4 4 5 6 6 8 7 source: 91.503 textbook Cormen et al. for Undirected, Connected, Weighted Graph G=(V,E) Produces minimum weight tree of edges that includes every vertex. Invariant: Minimum weight spanning forest Becomes single tree at end Invariant: Minimum weight tree Spans all vertices at end Time: O(|E|lg|E|) given fast FIND-SET, UNION Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue

17. Dijkstra Shortest Path

18. Single Source Shortest Paths: Dijkstra’s Algorithm source: 91.503 textbook Cormen et al. for (nonnegative) weighted, directed graph G=(V,E) A B C D E F G 2 2 1 1 3 4 4 5 6 6 8 7

19. Huffman Codes

20. source: 91.503 textbook Cormen, et al.

21. Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. source: web site accompanying 91.503 textbook Cormen, et al.

22. Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. source: web site accompanying 91.503 textbook Cormen, et al.

23. Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. source: web site accompanying 91.503 textbook Cormen, et al.

24. Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. source: web site accompanying 91.503 textbook Cormen, et al.

25. Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. source: web site accompanying 91.503 textbook Cormen, et al.

26. Fractional Knapsack

27. Knapsack 50 10 20 30 item1item2item3 “knapsack” Value: $60 $100$120 Each item has value and weight. Goal: maximize total value of items chosen, subject to weight limit. 0-1: take all or none of an item fractional: can take part of an item

28. Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. source: web site accompanying 91.503 textbook Cormen, et al.

29. Additional Examples On course web site under Miscellaneous Docs ä Patriotic Tree ä 404 review handout ä Tree Vertex Cover ä 91.503 midterm

30. Greedy Heuristic ä If optimization problem does not have “greedy choice property”, greedy approach may still be useful as a heuristic in bounding the optimal solution ä Example: minimization problem Optimal (unknown value) Upper Bound (heuristic) Lower Bound Solution Values

31. Homework 1 M 1/30 M 2/6 91.404 review & Chapter 15 2 M 2/6 T 2/13 Chapter 16 HW# Assigned Due Content