Greedy Algorithms Clayton Andrews 2/26/08

What is an algorithm? “An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output.” - (Cormen 5)‏

What is a greedy algorithm? A greedy algorithm is an algorithm that, at each step, is presented with choices, these choices are measured and one is determined to be the best and is selected. “It makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution” -(Cormen 370)‏

Greedy algorithms do Choose the largest, fastest, cheapest, etc... Typically make the problem smaller after each step or choice. Sometimes make decisions that turn out bad in the long run

Greedy algorithms don't Do not consider all possible paths Do not consider future choices Do not reconsider previous choices Do not always find an optimal solution

Greedy Algorithms Speed Because, greedy algorithms do not consider future decisions or reconsider any previous decisions, the time complexity of a greedy algorithm is usually magnitudes better than a brute force algorithm If a greedy algorithm is proven to find the optimal answer, always, the algorithm usually becomes standard because of speed and simplicity

Types of Solutions Produced by Greedy Algorithms Optimal Solutions – The best possible answer that any algorithm could find to the problem Good Solutions – A solution that is near optimal and could be good-enough for some problems Bad Solutions – A solution that is not acceptable Worst Possible Solution – The solution that is farthest from the goal

A simple problem Find the smallest number of coins whose sum reaches a specific goal Input: The total to reach and the coins usable Output: The smallest number of coins to reach the total

A greedy solution 1. Make a set with all types of coins 2. Choose the largest coin in set 3. If this coin will take the solution total over the target total, remove it from the set. Otherwise, add it to the solution set. 4. Calculate how large the current solution is 5. If the solution set sums up to the target total, a solution has been found, otherwise repeat 2-5

An Example Goal = 46 cents

Parts of a Greedy Algorithm A candidate set A selection function A feasibility function An objective function A solution function

Candidate Set The set the solution is constructed from. Members of the candidate set are checked to see if they can help construct the solution.

Candidate Set (Example)‏ Candidate Set Solution Set As you can see, solution set is made of members of the candidate set

Selection function The function that chooses what member of the candidate set to select The selection could be based off previous selections or some type of heuristic

Selection Function(Example)‏ The selection function for the change example chooses the largest coin in the candidate set As the coins are removed it still chooses the largest of the remaining. In this case, the selection function simply has to ensure the list is sorted and remove the largest.

Feasibility Function The function that determines if the choice of the selection function can help construct the solution The feasibility function may have to perform operations on the selected member to determine if all or part of it is acceptable

Feasibility Function (Example)‏ For the change example, the feasibility function determines if a coin is too large. If the difference between the solution and the goal is too small, then the selected coin is not feasible and is removed from the candidate set.

Objective Function The objective function determines the value of your current solution This value can be used to determine if the solution satisfies the requirements or if a member of the candidate set is feasible.

Objective Function(Example)‏ For the change example, the objective function scores the solution set In the example, the objective function just sums up the coins in the solution set. If the solution set had a quarter and a dime the objective function would return 35.

Solution Function The solution function determines if the current solution is a complete solution If the current solution meets the problem requirements If all choices are exhausted, then this must be a solution

Solution Function(Example)‏ In the change example, the solution function would compare the objective function's output with the target total. If the objective function's output is close enough to the target total, then the solution function ends the algorithm, because a solution has been found

Greedy Algorithm Part Summary Candidate Set – the set the solution is made from Selection Function – determines next candidate set member to be selected Feasibility Function – can the current candidate set be used to help construct the solution Objective Function – how good is the current solution Solution Function – is the current solution a complete solution

Examples Welsh-Powell Algorithm Kruskal's Algorithm 0/1 Knapsack

Welsh-Powell Algorithm The Welsh-Powell Algorithm is a greedy algorithm for finding good solutions to the graph coloring problem. Gives a good answer on average to the question “How many colors does it take to color this graph so that no neighbors have the same color?”

Welsh-Powell Algorithm Candidate Set – The nodes of the graph Selection Function – Remove node with the most edges Feasibility Function–Does this node have neighbor who is current color? If not, color the node Objective Function – Determine how many nodes are uncolored Solution Function – If the candidate set is not empty, repeat. If the candidate set is empty and the number of uncolored nodes is greater than 0 then choose a new color and put all uncolored nodes in the candidate set, then repeat

Kruskal's Algorithm Kruskal's algorithm finds the minimum spanning tree of a graph. This greedy algorithm always finds the optimal solution to the problem. Step 1 is to put each node into its own set

Kruskal's Algorithm Candidate Set – The edges of the graph Selection Function – Choose the smallest edge and remove it from the candidate set Feasibility Function – Does the selected edge form a cycle with its set. If not, merge the sets of both nodes of the edge and add the edge to the solution set Objective Function – Calculate how many sets are remaining, a complete solution has 1 Solution Function – If the number of sets is equal to 1, the solution is found, otherwise keep removing edges

0/1 Knapsack Given a series of items with values and weights and a knapsack with a maximum weight it can hold, what is the maximum value of items you can carry. Assume an item can only be added once The greedy solution guarantees a solution that is at least ½ the optimal solution. This isn't a very good guarantee, but good enough for some problems.

0/1 Knapsack Candidate Set – Items available to add Selection Function – Remove item with greatest Value/Weight ratio Feasibility Function – Does this item make the solution weight greater than the maximum capacity of the basket? If not, then add the item to the solution Objective Function – Find how much weight is left in the basket Solution Function – If there are items left in the candidate set and the basket can hold more weight then repeat, otherwise a solution has been found

Summary Greedy algorithms are fast and usually easy to make, which makes them very popular. Greedy algorithms have parts that are common amongst most of them. Although it is uncommon for a greedy algorithm to find an optimal solution, sometimes a pretty good solution is fine, because the algorithm is so fast.

Homework What are the 5 parts of a typical greedy algorithm? What is one thing that a greedy algorithm does not do?

Works Cited Introduction to Algorithms (Cormen, Leiserson, Rivest and Stein) (Just Images)