CSCE350 Algorithms and Data Structure Lecture 18 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.11.

Chapter 9: Greedy algorithms Change-making problem Coin-system in US: 25(quarter), 10 (dime), 5(nickel), 1(penny) If you need to give a change of 48 cents using coins, 48 cents = 1 quarter + 2 dimes + 3 pennies This is a greedy algorithm: reduce the amount in the fastest way The greedy approach constructs a solution through a sequence of steps until a complete solution is reached, On each step, the choice made must be Feasible: Satisfy the problem’s constraints locally optimal: the best choice Irrevocable: Once made, it cannot be changed later Not all optimization problems can be approached in this manner!

Applications of the Greedy Strategy Optimal solutions: change making Minimum Spanning Tree (MST) Single-source shortest paths simple scheduling problems Huffman codes Approximations: Traveling Salesman Problem (TSP) Knapsack problem other combinatorial optimization problems

Minimum Spanning Tree (MST) Motivation: Build a transportation system with the minimum length in a city  a tree structure (a connected acyclic graph) Spanning tree of a connected graph G: a connected acyclic subgraph of G that includes all of G’s vertices. Minimum Spanning Tree of a weighted, connected graph G: a spanning tree of G of minimum total weight. Example: 3 4 2 1 6 7 3 4 2 1 6 7

Prim’s MST algorithm Start with tree consisting of one vertex “grow” tree one vertex/edge at a time to produce MST Construct a series of expanding subtrees T1, T2, … at each stage construct Ti+1 from Ti: add minimum weight edge connecting a vertex in tree (Ti) to one not yet in tree choose from “fringe” edges (this is the “greedy” step!) algorithm stops when all vertices are included

An Example: Finding the MST of the following graph using Prim’s algorithm b d f a c 3 1 6 4 5 e 2 8

Step 1: Start from empty tree T, pick one vertex, a(-,-) and add it to T Remaining vertices: b(a,3), c(-,∞), d(-,∞), e(a,6), f(a,5) b d f a c 3 1 6 4 5 e 2 8

Step 2: Add the minimum-weight fringe edge b(a,3) into T Remaining vertices: c(b,1), d(-,∞), e(a,6), f(b,4) b d f a c 3 1 6 4 5 e 2 8

Step 3: Add the minimum-weight fringe edge c(b,1) into T Remaining vertices: d(c,6), e(a,6), f(b,4) b d f a c 3 1 6 4 5 e 2 8

Step 4: Add the minimum-weight fringe edge f(b,4) into T Remaining vertices: d(f,5), e(f,2) b d f a c 3 1 6 4 5 e 2 8

Step 5: Add the minimum-weight fringe edge e(f,2) into T Remaining vertices: d(f,5) b d f a c 3 1 6 4 5 e 2 8

Step 6: Add the minimum-weight fringe edge d(f,5) into T No remaining vertices and the algorithm is done! b d f a c 3 1 6 4 5 e 2 8

Does Prim’s Algorithm Really Produce MST? Lemma: Let X be any subset of the vertices of G, and let edge e be the smallest-weight edge connecting X (tree Ti-1) to G-X (remaining vertices). Then e (minimum-weight fringe edge) is part of the MST Proof: Using contradiction, suppose e=(u,v) is not part of MST. Then adding e to MST results in a cycle, which contains another edge e’=(u’,v’) between X and G-X. Replace e’ by e will result in a spanning tree with smaller total weight than MST. Contradiction!

Notes on Prim’s algorithm Pseudocode can be found in Section 9.1 To locate the minimum-weight fringe edge, we can use the heap structure. But here we use the min-heap, where the root has a key smaller than both children Delete the min – O(log |V|), it can to be performed |V|-1 times Verify minimum weight from any remaining vertex to the tree – this may be performed |E| times. Each verification may result in a key priority change in the heap, which takes O(log |V|). Therefore, the total complexity is (|V|-1+|E|)O(log |V|)=O(|E| log |V|)

Another Greedy algorithm for MST: Kruskal Start with empty forest of trees “grow” MST one edge at a time intermediate stages usually have forest of trees (not connected) at each stage add minimum weight edge among those not yet used that does not create a cycle edges are initially sorted by increasing weight at each stage the edge may: expand an existing tree combine two existing trees into a single tree create a new tree need efficient way of detecting/avoiding cycles algorithm stops when all vertices are included

An Example: Finding the MST of the following graph using Kruskal’s algorithm b d f a c 3 1 6 4 5 e 2 8

Step 1: Sorted List of Edges bc ef ab bf cf af df ae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

Step 2: add the next edge b d f a c 3 1 6 4 5 e 2 8 bc ef ab bf cf af ae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

Step 3: Add the next edge b d f a c 3 1 6 4 5 e 2 8 bc ef ab bf cf af ae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

Step 4: add the next edge b d f a c 3 1 6 4 5 e 2 8 bc ef ab bf cf af ae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

Step 5: add the next edge (without getting a cycle) bc ef ab bf cf af df ae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

Step 6: Finish the MST b d f a c 3 1 6 4 5 e 2 8 bc ef ab bf cf af df ae cd de 1 2 3 4 5 6 8 b d f a c 3 1 6 4 5 e 2 8

Does the Kruskal’s algorithm Produce MST Each newly added edge e=(u,v) is the minimum-weight edge between S (all the vertices that are reachable from u in the current forest) and the remaining vertices. According to the previous lemma, such an edge should be in MST.

Notes on Kruskal’s algorithm Algorithm looks easier than Prim’s but is harder to implement (checking for cycles!) less efficient Θ(|E| log |E|) Cycle checking: a cycle exists iff edge connects vertices in the same component. Union-find algorithms – see section 9.2