1. November 13, 20031 Algorithms and Data Structures Lecture XII Simonas Šaltenis Aalborg University simas@cs.auc.dk

2. November 13, 20032 This Lecture Weighted Graphs Minimum Spanning Trees Greedy Choice Theorem Kruskal’s Algorithm Prim’s Algorithm

3. November 13, 20033 Spanning Tree A spanning tree of G is a subgraph which is a tree contains all vertices of G How many edges are there in a spanning tree, if there are V vertices?

4. November 13, 20034 Minimum Spanning Trees Undirected, connected graph G = (V,E) Weight function W: E  R (assigning cost or length or other values to edges) Spanning tree: tree that connects all the vertices Optimization problem – Minimum spanning tree (MST): tree T that connects all the vertices and minimizes

5. November 13, 20035 Optimal Substructure ”Cut and paste” argument If G’ would have a cheaper ST T’, then we would get a cheaper ST of G: T’ + (u, v) MST(G) = T u v ”u+v” MST(G’) = T – (u,v)

6. November 13, 20036 Idea for an Algorithm We have to make V – 1 choices (edges of the MST) to arrive at the optimization goal After each choice we have a sub-problem one vertex smaller than the original Dynamic programming algorithm, at each stage, would consider all possible choices (edges) If only we could always guess the correct choice – an edge that definitely belongs to an MST

7. November 13, 20037 Greedy Choice Greedy choice property: locally optimal (greedy) choice yields a globally optimal solution Theorem Let G=(V, E), and let S  V and let (u,v) be min-weight edge in G connecting S to V – S : a light edge crossing a cut Then (u,v)  T – some MST of G

8. November 13, 20038 Greedy Choice (2) Proof Suppose (u,v) is light, but (u,v)  any MST look at path from u to v in some MST T Let (x, y) – the first edge on path from u to v in T that crosses from S to V – S. Swap (x, y) with (u,v) in T. this improves T – contradiction (T supposed to be an MST) u v x y S V-S

9. November 13, 20039 Generic MST Algorithm Generic-MST(G, w) 1 A  // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3 Find an edge (u,v) that is safe for A 4 A  A  {(u,v)} 5 return A Generic-MST(G, w) 1 A  // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3 Find an edge (u,v) that is safe for A 4 A  A  {(u,v)} 5 return A Safe edge – edge that does not destroy A’s property MoreSpecific-MST(G, w) 1 A  // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3.1 Make a cut (S, V-S) of G that respects A 3.2 Take the min-weight edge (u,v) connecting S to V-S 4 A  A  {(u,v)} 5 return A MoreSpecific-MST(G, w) 1 A  // Contains edges that belong to a MST 2 while A does not form a spanning tree do 3.1 Make a cut (S, V-S) of G that respects A 3.2 Take the min-weight edge (u,v) connecting S to V-S 4 A  A  {(u,v)} 5 return A

10. November 13, 200310 Pseudocode assumptions Graph ADT with an operation V():VertexSet E(): EdgeSet w(u:Vertex, v:Vertex):int – returns a weight of (u,v) A looping construct “for each v  V ”, where V is of a type VertexSet, and v is of a type Vertex Vertex ADT with operations: adjacent():VertexSet key():int and setkey(k:int) parent():Vertex and setparent(p:Vertex)

11. November 13, 200311 Prim-Jarnik Algorithm Vertex based algorithm Grows one tree T, one vertex at a time A set A covering the portion of T already computed Label the vertices v outside of the set A with v.key() – the minimum weigth of an edge connecting v to a vertex in A, v.key() = , if no such edge exists

12. November 13, 200312 MST-Prim(G,r) 01 for each vertex u  G.V() 02 u.setkey(  03 u.setparent(NIL) 04 r.setkey(0) 05 Q.init(G.V()) // Q is a priority queue ADT 06 while not Q.isEmpty() 07 u  Q.extractMin() // making u part of T 08 for each v  u.adjacent() do 09 if v  Q and G.w(u,v) < v.key() then 10 v.setkey(G.w(u,v)) 11 Q.modifyKey(v) 12 v.setparent(u) Prim-Jarnik Algorithm (2) updating keys

13. November 13, 200313 Prim-Jarnik’s Example Let’s do an example: HBCIADGFE Let’s keep track: What is set A, which vertices are in Q, what cuts are we making, what are the key values 1 6 8 4 12 8 3 6 5 3 4 13 9 10

14. November 13, 200314 MST-Prim(G,r) 01 for each vertex u  G.V() 02 u.setkey(  03 u.setparent(NIL) 04 r.setkey(0) 05 Q.init(G.V()) // Q is a priority queue ADT 06 while not Q.isEmpty() 07 u  Q.extractMin() // making u part of T 08 for each v  u.adjacent() do 09 if v  Q and G.w(u,v) < v.key() then 10 v.setkey(G.w(u,v)) 11 Q.modifyKey(v) 12 v.setparent(u) Implementation issues

15. November 13, 200315 Priority Queues A priority queue is an ADT for maintaining a set S of elements, each with an associated value called key We need PQ to support the following operations init(S:VertexSet) extractMin():Vertex modifyKey(v:Vertex) To choose how to implement a PQ, we need to count how many times the operations are performed: init is performed just once and runs in O(n)

16. November 13, 200316 Prim-Jarnik’s Running Time Time = |V|T(extractMin) + O(E)T(modifyKey) Binary heap implementation: Time = O(V lgV + E lgV) = O(E lgV) QT(extractMin)T(modifyKey)Total arrayO(V)O(V)O(1)O(V 2 ) binary heapO(lg V) O(E lgV ) Fibonacci heap O(lg V)O(1) amortizedO(V lgV +E )

17. November 13, 200317 Greedy Algorithms One more algorithm design technique Greedy algorithms can be used to solve optimization problems, if: There is an optimal substructure We can prove that a greedy choice at each iteration leads to an optimal solution.

18. November 13, 200318 Kruskal's Algorithm Edge based algorithm Add the edges one at a time, in increasing weight order The algorithm maintains A – a forest of trees. An edge is accepted it if connects vertices of distinct trees (the cut respects A)

19. November 13, 200319 Disjoint-Set ADT We need a Disjoint-Set ADT that maintains a partition, i.e., a collection S of disjoint sets. Operations: makeSet(x:Vertex): SetID S  S  {{x}} union(x:Vertex, y:Vertex) – X  S.findSet(x) Y  S.findSet(y) S  S – {X, Y}  {X  Y} findSet(x:Vertex): SetID returns unique X  S, where x  X

20. November 13, 200320 Kruskal's Algorithm The algorithm keeps adding the cheapest edge that connects two trees of the forest MST-Kruskal(G) 01 A   02 S.init() // Init disjoint-set ADT 03 for each vertex v  G.V() 04 S.makeSet(v) 05 sort the edges of G.E() by non-decreasing G.w(u,v) 06 for each edge (u,v)  E, in sorted order do 07 if S.findSet(u)  S.findSet(v) then 08 A  A  {(u,v)} 09 S.union(u,v) 10 return A

21. November 13, 200321 Kruskal’s Example Let’s do an example: HBCIADGFE Let’s keep track: What is set A, what is a collection S, what cuts are we making 1 6 8 4 12 8 3 6 5 3 4 13 9 10

22. November 13, 200322 Disjoint Sets as Lists Each set – a list of elements identified by the first element, all elements in the list point to the first element Union – add a smaller list to a larger one FindSet: O(1), Union(u,v): O(min{|C(u)|, |C(v)|}) 123 ABC  4  123 ABC  4

23. November 13, 200323 Kruskal Running Time Initialization O(V) time Sorting the edges  (E lg E) =  (E lg V) (why?) O(E) calls to FindSet Union costs Let t(v) be the number of times v is moved to a new cluster Each time a vertex is moved to a new cluster the size of the cluster containing the vertex at least doubles: t(v)  log V Total time spent doing Union Total time: O(E lg V)

24. November 13, 200324 Next Lecture Shortest Paths in Weighted Graphs