CS216: Program and Data Representation University of Virginia Computer Science Spring 2006 David Evans Lecture 7: Greedy Algorithms

Slides:



Advertisements
Similar presentations
CS Section 600 CS Section 002 Dr. Angela Guercio Spring 2010.
Advertisements

MCS 312: NP Completeness and Approximation algorithms Instructor Neelima Gupta
Introduction to Algorithms Greedy Algorithms
Chapter 5 Fundamental Algorithm Design Techniques.
Analysis of Algorithms
Greed is good. (Some of the time)
Greedy Algorithms Be greedy! always make the choice that looks best at the moment. Local optimization. Not always yielding a globally optimal solution.
Greedy Algorithms Basic idea Connection to dynamic programming
Greedy Algorithms Basic idea Connection to dynamic programming Proof Techniques.
Merge Sort 4/15/2017 6:09 PM The Greedy Method The Greedy Method.
Greedy vs Dynamic Programming Approach
Lecture 7: Greedy Algorithms II Shang-Hua Teng. Greedy algorithms A greedy algorithm always makes the choice that looks best at the moment –My everyday.
Greedy Algorithms CIS 606 Spring Greedy Algorithms Similar to dynamic programming. Used for optimization problems. Idea – When we have a choice.
CSE 421 Algorithms Richard Anderson Lecture 6 Greedy Algorithms.
Week 2: Greedy Algorithms
Lecture 7: Greedy Algorithms II
1 The Greedy Method CSC401 – Analysis of Algorithms Lecture Notes 10 The Greedy Method Objectives Introduce the Greedy Method Use the greedy method to.
10/31/02CSE Greedy Algorithms CSE Algorithms Greedy Algorithms.
9/3/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Guest lecturer: Martin Furer Algorithm Design and Analysis L ECTURE.
10/31/02CSE Greedy Algorithms CSE Algorithms Greedy Algorithms.
Scott Perryman Jordan Williams.  NP-completeness is a class of unsolved decision problems in Computer Science.  A decision problem is a YES or NO answer.
9/8/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Adam Smith Algorithm Design and Analysis L ECTURE 6 Greedy Algorithms.
Design of Algorithms using Brute Force Approach. Primality Testing (given number is n binary digits)
CSCI-256 Data Structures & Algorithm Analysis Lecture Note: Some slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved. 8.
Algorithmics - Lecture 101 LECTURE 10: Greedy technique.
Greedy algorithms David Kauchak cs161 Summer 2009.
5-1-1 CSC401 – Analysis of Algorithms Chapter 5--1 The Greedy Method Objectives Introduce the Brute Force method and the Greedy Method Compare the solutions.
Lectures on Greedy Algorithms and Dynamic Programming
CSCI 256 Data Structures and Algorithm Analysis Lecture 6 Some slides by Kevin Wayne copyright 2005, Pearson Addison Wesley all rights reserved, and some.
The Greedy Method. The Greedy Method Technique The greedy method is a general algorithm design paradigm, built on the following elements: configurations:
CSC 201: Design and Analysis of Algorithms Greedy Algorithms.
Class 24: P=NP? David Evans cs302: Theory of Computation University of Virginia Computer Science PS6 (the last one) is.
December 14, 2015 Design and Analysis of Computer Algorithm Pradondet Nilagupta Department of Computer Engineering.
1 Chapter 5-1 Greedy Algorithms Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved.
Dynamic Programming.  Decomposes a problem into a series of sub- problems  Builds up correct solutions to larger and larger sub- problems  Examples.
Lecture 8 CSE 331. Main Steps in Algorithm Design Problem Statement Algorithm Problem Definition “Implementation” Analysis n! Correctness+Runtime Analysis.
Greedy Algorithms BIL741: Advanced Analysis of Algorithms I (İleri Algoritma Çözümleme I)1.
Greedy Algorithms Interval Scheduling and Fractional Knapsack These slides are based on the Lecture Notes by David Mount for the course CMSC 451 at the.
CS216: Program and Data Representation University of Virginia Computer Science Spring 2006 David Evans Lecture 8: Crash Course in Computational Complexity.
Greedy algorithms David Kauchak cs302 Spring 2012.
Spring 2008The Greedy Method1. Spring 2008The Greedy Method2 Outline and Reading The Greedy Method Technique (§5.1) Fractional Knapsack Problem (§5.1.1)
1 Algorithms CSCI 235, Fall 2015 Lecture 29 Greedy Algorithms.
Ch3 /Lecture #4 Brute Force and Exhaustive Search 1.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 17.
Introduction to Algorithms: Brute-Force Algorithms.
Greedy Algorithms Many of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill.
Greedy Algorithms Many of the slides are from Prof. Plaisted’s resources at University of North Carolina at Chapel Hill.
Greedy Algorithms.
Lecture on Design and Analysis of Computer Algorithm
Weighted Interval Scheduling
Merge Sort 7/29/ :21 PM The Greedy Method The Greedy Method.
Greedy Algorithms Basic idea Connection to dynamic programming
Greedy Algorithms / Interval Scheduling Yin Tat Lee
Presented by Po-Chuan & Chen-Chen 2016/03/08
Merge Sort 11/28/2018 2:18 AM The Greedy Method The Greedy Method.
The Greedy Method Spring 2007 The Greedy Method Merge Sort
Merge Sort 11/28/2018 2:21 AM The Greedy Method The Greedy Method.
Merge Sort 11/28/2018 8:16 AM The Greedy Method The Greedy Method.
Advanced Analysis of Algorithms
Advanced Algorithms Analysis and Design
Merge Sort 1/17/2019 3:11 AM The Greedy Method The Greedy Method.
Richard Anderson Lecture 6 Greedy Algorithms
Lecture 19 CSE 331 Oct 8, 2014.
Lecture 18 CSE 331 Oct 9, 2017.
Richard Anderson Lecture 7 Greedy Algorithms
Algorithms CSCI 235, Spring 2019 Lecture 29 Greedy Algorithms
Merge Sort 5/2/2019 7:53 PM The Greedy Method The Greedy Method.
Week 2: Greedy Algorithms
Lecture 19 CSE 331 Oct 10, 2016.
Richard Anderson Autumn 2019 Lecture 7
Presentation transcript:

CS216: Program and Data Representation University of Virginia Computer Science Spring 2006 David Evans Lecture 7: Greedy Algorithms

2 UVa CS216 Spring Lecture 7: Greed is Good Menu “Life is uncertain. Eat dessert first.” Ernestine Ulmer La vita e incerta - mangia il dolce per primo.

3 UVa CS216 Spring Lecture 7: Greed is Good Greed is Good? Adam Smith, An Inquiry into the Nature and Causes of the Wealth of Nations (1776) “It is not from the benevolence of the butcher, the brewer, or the baker, that we expect our dinner, but from their regard to their own interests.” Invisible hand: individuals acting on personal greed produces (nearly) globally optimal results

4 UVa CS216 Spring Lecture 7: Greed is Good Greedy Algorithms Make the locally best choice “myopically” at each step –Need to figure out what “dessert” is Hope it leads to a globally good solution –Sometimes, can prove it leads to an optimal solution –Other times (like phylogeny), non- optimal, but usually okay if you get lucky

5 UVa CS216 Spring Lecture 7: Greed is Good Interval Scheduling Problem Input: R, a set of n resource requests: {,, …, } Output: a subset S of R with no overlapping requests ( s i > s j,  S ) such that |S|  |T| for any T  R with no overlapping requests

6 UVa CS216 Spring Lecture 7: Greed is Good Example R = {,,,,, }

7 UVa CS216 Spring Lecture 7: Greed is Good Solution R = {,,,,, }

8 UVa CS216 Spring Lecture 7: Greed is Good Brute Force Algorithm Try all possible subsets –Filter out ones with overlapping intervals –Pick the largest subset Running time –How many subsets? –Constant work for each subset – (2 n ) 2n2n

9 UVa CS216 Spring Lecture 7: Greed is Good Greedy Approaches Need to pick best subset by making myopic decisions, one element at a time Many possible criteria for making myopic decision –Earliest starting time? –Latest ending time? –Shortest?

10 UVa CS216 Spring Lecture 7: Greed is Good Greedy Approach: Earliest Starting Not optimal: |S| < |best| = 3

11 UVa CS216 Spring Lecture 7: Greed is Good Greedy Approach: Earliest Finishing Not optimal: |S| < |best| = 3

12 UVa CS216 Spring Lecture 7: Greed is Good Greedy Approach: Shortest Length ?

13 UVa CS216 Spring Lecture 7: Greed is Good Greedy Approach: Shortest Length R = {,, } Not optimal: |S| < |best| = 2

14 UVa CS216 Spring Lecture 7: Greed is Good Greedy Approach: Pick Earliest Finishing Time

15 UVa CS216 Spring Lecture 7: Greed is Good Greedy Algorithm: Running Time Analysis Straightforward implementation: –Search to find earliest finishing: O(n) –Eliminate matching elements: O(n) –Repeat (up to n times): O(n 2 ) Smarter implementation: –Sort by finishing time: O(n log n) –Go through list, selecting if non- overlapped: O(n) –Running time  O(n log n)

16 UVa CS216 Spring Lecture 7: Greed is Good Correctness? How to prove a greedy algorithm is nonoptimal –Find a counterexample: some input where the greedy algorithm does not find the best solution How to prove a greedy algorithm is optimal –By induction: always best up to some size –By exchange argument: swapping any element in solution cannot improve result

17 UVa CS216 Spring Lecture 7: Greed is Good Proof The greedy algorithm produces, R = { r 0, …, r k-1 } Suppose there is a better subset, Q = { q 0, …, q k-1, q k } Sort both by finishing time, so f r i < f r j for all 0  i < j < k f q i < f q j for all 0  i < j < k+1

18 UVa CS216 Spring Lecture 7: Greed is Good Proof R = { r 0, …, r k-1 } Q = { q 0, …, q k-1, q k } f r i < f r j for all 0  i < j < k f q i < f q j for all 0  i < j < k+1 Strategy: 1. Prove by induction f r i  f q j for all i < k 2. Then, since f r k-1  f q k-1 if q k is valid, it would have also been added to R.

19 UVa CS216 Spring Lecture 7: Greed is Good Induction Proof: f r i  f q j Basis: f r 0  f q 0 –Greedy algorithm choose r 0 as the element with the earliest finishing time –So, f r 0  f j for all j Induction: f r i-1  f q j-1  f r i  f q j –Since f r i-1  f q j-1 we know s q i  f r i-1 –So, greedy algorithm could choose q i –If f q i < f r j, greedy algorithm would have chosen f q j instead of f r j R = { r 0, …, r k-1 } Q = { q 0, …, q k-1, q k }

20 UVa CS216 Spring Lecture 7: Greed is Good Knapsack Problems You have a collection of items, each has a value and weight How to optimally fill a knapsack with as many items as you can carry Scheduling: weight = time, one deadline for all tasks Budget allocation: weight = cost

21 UVa CS216 Spring Lecture 7: Greed is Good General Knapsack Problem Input: a set of n items {, …, }, and maxweight Output: a subset of the input items such that the sum of the weights of all items in the output set is  maxweight and there is no subset with weight sum  maxweight with a greater value sum

22 UVa CS216 Spring Lecture 7: Greed is Good Brute Force Knapsack def knapsack (items, maxweight): best = {} bestvalue = 0 for s in allPossibleSubsets (items): value = 0 weight = 0 for item in s: value += item.value weight += item.weight if weight <= maxweight: if value > bestvalue: best = s bestvalue = value return best (Defining and analyzing this might be a good Exam 1 question)

23 UVa CS216 Spring Lecture 7: Greed is Good Brute Force Knapsack Analysis How many subsets are there? How much work for each subset? 2n2n for item in s: value += item.value weight += item.weight Average size of each subset is n /2 (there are as many subsets with size c and of size n – c ) Running time  ( n2 n )

24 UVa CS216 Spring Lecture 7: Greed is Good Dynamic Programming Section this week: dynamic programming solution to the knapsack problem Running time in O(maxweight * n)

25 UVa CS216 Spring Lecture 7: Greed is Good Greedy Knapsack Algorithm Repeat until no more items fit: –Add the most valuable item that fits “Greedy”: always picks the most valuable item that fits first

26 UVa CS216 Spring Lecture 7: Greed is Good Greedy Knapsack Algorithm def knapsack_greedy (items, maxweight): result = [] weight = 0 while True: # try to add the best item weightleft = maxweight - weight bestitem = None for item in items: if item.weight <= weightleft \ and (bestitem == None \ or item.value > bestitem.value): bestitem = item if bestitem == None: break else: result.append (bestitem) weight += bestitem.weight return result Running Time  (n 2 )

27 UVa CS216 Spring Lecture 7: Greed is Good Is Greedy Algorithm Correct? No. Proof by counterexample: Consider input items = {, } maxweight = 3 Greedy algorithm picks { } value = 110, but {, “silver”>} has weight <= 3 and value = 180

28 UVa CS216 Spring Lecture 7: Greed is Good Same Weights Knapsack Problem Input: a set of n items {, …, }, each having weight w, and maxweight Output: a subset of the input items such that the sum of the weights of all items in the output set is  maxweight and there is no subset with weight sum  maxweight with a greater value sum

29 UVa CS216 Spring Lecture 7: Greed is Good Greedy Algorithm Correct It keeps adding items until maxweight would be exceeded, so the result contains k items where kw maxweight Hence, cannot add any item (weight w ) without removing another item But, any item with value > value of the lowest value item in result would have already been added by greedy algorithm

30 UVa CS216 Spring Lecture 7: Greed is Good Subset Sum Problem Knapsack problem where value = weight Input: set of n positive integers, {w 0, …, w n-1 }, maximum weight W Output: a subset S of the input set such that the sum of the elements of S  W and there is no subset of the input set whose sum is greater than the sum of S and  W

31 UVa CS216 Spring Lecture 7: Greed is Good Brute Force Subset Sum def subsetsum (items, maxweight): best = {} for s in allPossibleSubsets (items): if sum (s) <= maxweight \ and sum (s) > sum (best) best = s return best Just like brute force knapsack: Running time  ( n2 n )

32 UVa CS216 Spring Lecture 7: Greed is Good Greedy Subset Sum? Pick largest item that fits –Bad: I = { 4, 5, 7 } W = 9 Pick smallest item –Bad: I = { 4, 5, 7 } W = 7 Doesn’t prove there is no myopic criteria that works Note: Subset Sum is known to be NP- Complete, so finding one would prove P = NP

33 UVa CS216 Spring Lecture 7: Greed is Good Charge More greedy algorithm examples in Section this week PS3: greedy phylogeny algorithm –Not optimal (prove in Question 8) –Usually reasonably good (similar to algorithms used in practice)