Lecture 19 Greedy Algorithms Minimum Spanning Tree Problem

Example: Coin Changing Problem Give change using the minimum number of coins. Greedy approach: always use larger coins first Example when it doesn’t work: 16c due, coins of 12, 10, 5 and 1. Greedy: 12, 1, 1, 1, 1 Optimal: 10, 5, 1 Moral: need to prove that greedy produces optimal

Greedy Algorithm Applies to Optimization Problems (find minimum or maximum such that…) Strategy: construct the solution by choosing elements one at a time in a greedy fashion (local optimum item), hoping that it will produce the global optimum structure.

Minimum Spanning Tree Given undirected connected weighted graph (“costs” assigned to edges). Spanning tree: minimum connected subgraph Minimum weight (or cost) spanning tree: sum over all edges is minimum over all possible spanning trees

Optimum Spanning Tree Minimum Maximum

Naïve Solution Enumerate all trees, compute their costs Find minimum (resp. maximum) Why bad? Number of trees on n vertices is very large (exponential): n n-2. Hence number of spanning trees can be very large => inefficient algorithm (exponential)

Greedy Algorithms for MST Prim’s algorithm: start with some vertex v. At next step, choose the minimum edge adjacent to already constructed tree, while maintaining the tree property Kruskal’s algorithm: sort edges in increasing order, add one at a time while maintaining forest property

Prim’s Algorithm for MST

Prim’s Algorithm: Invariant At each step maintain a tree Compute fringe: list of all edges out of the current tree Find minimum edge in the fringe Add it to the tree

Prim’s Algorithm: Analysis At each step add a vertex: n steps Compute fringe: list of all edges out of the current tree Find minimum edge in the fringe: O(m) Add it to the tree Total: O(nm) Naive

Prim’s Algorithm: Better Implementation and Analysis At each step add a vertex: n steps Compute fringe: list of all edges out of the current tree, keep as heap. Must look at all edges O(m) Find minimum edge in the fringe: O(log m) = O(log n) Add it to the tree Total: O(m+nlog n)

Kruskal’s Algorithm for MST Sort all edges O(m 2 log n) At each step, choose the next min weight edge from the list that does not create a cycle. Invariant: maintain a forest of the min weight edges encountered up to step k

Kruskal’s Algorithm for MST

Kruskal’s Algorithm for MST: forest along the way

Kruskal’s Algorithm for MST: Analysis Sort all edges O(m 2 log n) At each step, choose the next min weight edge from the list that does not create a cycle: O(log n). Total n-1 steps Total: O(m 2 log n)

Correctness Must prove that the greedy strategy produces the optimum solution Prim: show that if the min edge is not chosen at some step k, then it is not a MST (there exists a tree of smaller cost) Kruskal: show that min edge must be in MST; then next one, and next one etc.

Correctness of Prim’s Algorithm Assume MST unique By contradiction. Show that if the min fringe edge is not chosen at some step k, then it is not a MST (there exists a tree of smaller cost)

Prim’s Algorithm: Correctness Assume turquoise tree is MST but does not use red edge (min at this step) Red edge forms a cycle with tree edges, with at least one other edge in the fringe Removing this edge of the cycle and putting back the red edge creates a smaller cost tree Contradiction